If you apply a test like this to a few million people you will get tens of thousands of false positives. Those will clog up the works while in the meantime your quiet unassuming terrorist will walk right by under the radar and quite possibly will kill a bunch of people.
Most blood tests or tests for any major disease have a high rate of false positive, that's why you take the tests ALONG with a doctor analysing your symptoms (and that's there Bayes' theorem plays a big role).
So applying a single test to a huge general population makes little sense. But applying that to a population that already carries some traits is highly relevant.
Blood tests for diseases tend to have a much higher true positive rate as well because of substantial higher incidence rate of diseases for which simple blood tests exist than terrorism, besides that the cost of a false positive isn't that high (you just order another test to improve the odds, roughly at the same cost level as the first one).
Even Bayes can't make sense of garbage input, there has to be some relevance to the input numbers.
It's not magic, it's applied statistics.
If your probabilities are on the order of 'nonexistent' and your test has a false positive rate on the order of a few 10'ths of a % then you are wasting your time no matter what the method you use to analyze your data.
If your metric is simply "number of terrorists caught" then sure a broken test is better than no test.
But you are ignoring the cost of "catching" innocent people. Plus you are ignoring the cost of the administering the test.
You could dedicate your resources to actual detective work instead of wholesale dragnet operation in the hope that something useful turns up. Makes for good security theater though.
Uhm, what? The alternative is to have a few million people that you're concerned about for some reason and then have no way to exclude the vast majority of them from investigative interest.
The test is useful, just not useful enough to transform a random population into a list of suspects. It's useful for what the FBI says it's getting used for - determining how to allocate resources among a limited population that's already under suspicion. Eg, be more likely to assign an active agent if the case involves people with access to weapons.
> The alternative is to have a few million people that you're concerned about for some reason and then have no way to exclude the vast majority of them from investigative interest.
If you're concerned about a few million people to spot 10 (a high estimate) per year then you will need better tools than dragnet surveillance and questionnaires like these. Did you really expect to answer any of these questions about a would-be terrorist in such a way that it would help an investigation?
Check the questions, it's positively skewed towards one particular kind of terrorist (young Islamic males even though it does not state that outright it might as well be written there). And then you get your nice, upstanding Christian white-guy who murders 6 people in cold blood and would have murdered a whole lot more if he had the chance.
These questions are meaningless without accurate priors and accurate priors require incidence rates that are distinguishable from noise.
