Leaked NSA slide-deck claims that NSA has “direct access” to servers at Google, Apple, Facebook, Skype, Yahoo, and many others

This story has received a lot of coverage in the last few days, even extending to Jay Leno’s monologue. Understandably, there is a lot of concern about the civil liberties implications. But there is also another question, answered by Corey Chivers: How likely is the NSA PRISM program to catch a terrorist?

We don’t really know anything about how PRISM works (NSA = Never Say Anything), but with some plausible assumptions we can estimate the answer. Suppose

1. If a terrorist is in the system, the probability is 99% that PRISM will flag him/her.
2. An good guy has only a 1/100 chance of being flagged as potential terrorist.
3. The actual number of terrorists is quite small, say 1 in 1,000,000.

Using Bayes’ rule, Chivers shows that only 1 in 10,102 of the people flagged as suspects will actually be a terrorist!

The key point is that assumption (3) completely dominates the analysis. Unless the tests are very accurate, the terrorists will be thoroughly hidden in the false positives.

Actually, the problem is simple enough that you do not need to explicitly use Bayes’ Theorem:

Suppose you have a population of 100,000,000 people. Then by assumption (3) you can expect 100 bad guys and 100,000,000 – 100 = 99,999,900 good guys.

The test is 99% accurate at detecting the bad guys so it will flag 99 of them.

The test has a 1% probability of flagging an innocent person, so 0.01 x 99,999,900 = 999,999 good guys will be flagged as suspects.

Hence 99 + 999,999 = 1000098 people will be flagged as suspects.

Of those suspects only 99/1000098 = 9.89902989507028E-0058 = 1/10,102 will be terrorists, agreeing with the result derived by Chivers.