### Friday, May 26, 2006

## Instilling Paranoia: It's An American Value

Here’s a serious analysis of what’s happening with the government’s program of gathering telephone information. Warning: if you start looking at the Wikipedia article on Bayes’ Theorem, you will get a headache and your eyes will cross.

This came, by the way, from my good friend George H., who I hope hasn’t looked at Bayes Theorem.

[Bayes' Theorem, (http://en.wikipedia.org/wiki/Bayes%27_Theorem) an

important 18th-century advance in the understanding of probability, tells

how to update or revise beliefs in light of new evidence. -- On Wednesday,

Prof. Floyd Rudmin of the University of Tromso, Norway, used Bayes' Theorem

to demonstrated that "mass surveillance of an entire population cannot find

terrorists. It is a probabilistic impossibility. It cannot work."[1] --

Moreover, Rudmin argues that since "[e]veryone at NSA certainly knows Bayes'

Theorem," the only plausible reasons for the spying that NSA is doing are

either (1) paranoia, or 2) political espionage. -- Q.E.D. --

http://www.ufppc.org/content/view/4536/

1.

THE POLITICS OF PARANOIA AND INTIMIDATION

By Floyd Rudmin

** Why Does the NSA Engage in Mass Surveillance of Americans When It's

Statistically Impossible for Such Spying to Detect Terrorists? **

CounterPunch

May 24, 2006

http://www.counterpunch.org/rudmin05242006.html

The Bush administration and the National Security Agency (NSA) have been

secretly monitoring the email messages and phone calls of all Americans.

They are doing this, they say, for our own good. To find terrorists. Many

people have criticized NSA's domestic spying as unlawful invasion of

privacy, as search without search warrant, as abuse of power, as misuse of

the NSA's resources, as unconstitutional, as something the Communists would

do, something very un-American.

In addition, however, mass surveillance of an entire population cannot find

terrorists. It is a probabilistic impossibility. It cannot work.

What is the probability that people are terrorists given that NSA's mass

surveillance identifies them as terrorists? If the probability is zero

(p=0.00), then they certainly are not terrorists, and NSA was wasting

resources and damaging the lives of innocent citizens. If the probability

is one (p=1.00), then they definitely are terrorists, and NSA has saved the

day. If the probability is fifty-fifty (p=0.50), that is the same as

guessing the flip of a coin. The conditional probability that people are

terrorists given that the NSA surveillance system says they are, that had

better be very near to one (p=1.00) and very far from zero (p=0.00).

The mathematics of conditional probability were figured out by the Scottish

logician Thomas Bayes. If you Google "Bayes' Theorem ", you will get more

than a million hits. Bayes' Theorem is taught in all elementary statistics

classes. Everyone at NSA certainly knows Bayes' Theorem.

To know if mass surveillance will work, Bayes' theorem requires three

estimations:

1) The base-rate for terrorists, i.e. what proportion of the population are

terrorists.

2) The accuracy rate, i.e., the probability that real terrorists will be

identified by NSA;

3) The misidentification rate, i.e., the probability that innocent citizens

will be misidentified by NSA as terrorists.

No matter how sophisticated and super-duper are NSA's methods for

identifying terrorists, no matter how big and fast are NSA's computers,

NSA's accuracy rate will never be 100% and their misidentification rate will

never be 0%. That fact, plus the extremely low base-rate for terrorists,

means it is logically impossible for mass surveillance to be an effective

way to find te rrorists.

I will not put Bayes' computational formula here. It is available in all

elementary statistics books and is on the web should any readers be

interested. But I will compute some conditional probabilities that people

are terrorists given that NSA's system of mass surveillance identifies them

to be terrorists.

The U.S. Census shows that there are about 300 million people living in the

USA.

Suppose that there are 1,000 terrorists there as well, which is probably a

high estimate. The base-rate would be 1 terrorist per 300,000 people. In

percentages, that is .00033% which is way less than 1%. Suppose that NSA

surveillance has an accuracy rate of .40, which means that 40% of real

terrorists in the USA will be identified by NSA's monitoring of everyone's

email and phone calls. This is probably a high estimate, considering that

terrorists are doing their best to avoid detection. There is no evidence

thus far t hat NSA has been so successful at finding terrorists. And suppose

NSA's misidentification rate is .0001, which means that .01% of innocent

people will be misidentified as terrorists, at least until they are

investigated, detained, and interrogated. Note that .01% of the US

population is 30,000 people. With these suppositions, then the probability

that people are terrorists given that NSA's system of surveillance

identifies them as terrorists is only p=0.0132, which is near zero, very far

from one. Ergo, NSA's surveillance system is useless for finding

terrorists.

Suppose that NSA's system is more accurate than .40, let's say, .70, which

means that 70% of terrorists in the USA will be found by mass monitoring of

phone calls and email messages. Then, by Bayes' Theorem, the probability

that a person is a terrorist if targeted by NSA is still only p=0.0228,

which is near zero, far from one, and useless.

Suppose that NSA's system is really, really, really good, with an accuracy

rate of .90, and a misidentification rate of .00001, which means that only

3,000 innocent people are misidentified as terrorists. With these

suppositions, then the probability that people are terrorists given that

NSA's system of surveillance identifies them as terrorists is only p=0.2308,

which is far from one and well below flipping a coin. NSA's domestic

monitoring of everyone's email and phone calls is useless for finding

terrorists.

NSA knows this. Bayes' Theorem is elementary common knowledge. So, why

does NSA spy on Americans knowing it's not possible to find terrorists that

way? Mass surveillance of the entire population is logically sensible only

if there is a higher base-rate. Higher base-rates arise from two lines of

thought, neither of them very nice:

1) McCarthy-type national paranoia;

2) political espionage.

The whole NSA domestic spying program will seem to work well, will seem

logical and possible, if you are paranoid. Instead of presuming there are

1,000 terrorists in the USA, presume there are 1 million terrorists.

Americans have gone paranoid before, for example, during the McCarthyism era

of the 1950s. Imagining a million terrorists in America puts the base-rate

at .00333, and now the probability that a person is a terrorist given that

NSA's system identifies them is p=.99, which is near certainty. But only if

you are paranoid. If NSA's surveillance requires a presumption of a million

terrorists, and if in fact there are only 100 or only 10, then a lot of

innocent people are going to be misidentified and confidently mislabeled as

terrorists.

The ratio of real terrorists to innocent people in the prison camps of

Guantanamo, Abu Ghraib, and Kandahar shows that the U.S. is paranoid and is

not bothered by mistaken identifications of innocent people. The rati o of

real terrorists to innocent people on Bush's no-fly lists shows that the

Bush administration is not bothered by mistaken identifications of innocent

Americans.

Also, mass surveillance of the entire population is logically plausible if

NSA's domestic spying is not looking for terrorists, but looking for

something else, something that is not so rare as terrorists. For example,

the May 19 Fox News opinion poll of 900 registered voters found that 30%

dislike the Bush administration so much they want him impeached. If NSA

were monitoring email and phone calls to identify pro-impeachment people,

and if the accuracy rate were .90 and the error rate were .01, then the

probability that people are pro-impeachment given that NSA surveillance

system identified them as such, would be p=.98, which is coming close to

certainty (p_1.00). Mass surveillance by NSA of all Americans' phone calls

and emails would be very effective for dom estic political intelligence.

But finding a few terrorists by mass surveillance of the phone calls and

email messages of 300 million Americans is mathematically impossible, and

NSA certainly knows that.

--Floyd Rudmin is Professor of Social & Community Psychology at the

University of Tromsø in Norway. He can be reached at frudmin@psyk.uit.no

This came, by the way, from my good friend George H., who I hope hasn’t looked at Bayes Theorem.

[Bayes' Theorem, (http://en.wikipedia.org/wiki/Bayes%27_Theorem) an

important 18th-century advance in the understanding of probability, tells

how to update or revise beliefs in light of new evidence. -- On Wednesday,

Prof. Floyd Rudmin of the University of Tromso, Norway, used Bayes' Theorem

to demonstrated that "mass surveillance of an entire population cannot find

terrorists. It is a probabilistic impossibility. It cannot work."[1] --

Moreover, Rudmin argues that since "[e]veryone at NSA certainly knows Bayes'

Theorem," the only plausible reasons for the spying that NSA is doing are

either (1) paranoia, or 2) political espionage. -- Q.E.D. --

http://www.ufppc.org/content/view/4536/

1.

THE POLITICS OF PARANOIA AND INTIMIDATION

By Floyd Rudmin

** Why Does the NSA Engage in Mass Surveillance of Americans When It's

Statistically Impossible for Such Spying to Detect Terrorists? **

CounterPunch

May 24, 2006

http://www.counterpunch.org/rudmin05242006.html

The Bush administration and the National Security Agency (NSA) have been

secretly monitoring the email messages and phone calls of all Americans.

They are doing this, they say, for our own good. To find terrorists. Many

people have criticized NSA's domestic spying as unlawful invasion of

privacy, as search without search warrant, as abuse of power, as misuse of

the NSA's resources, as unconstitutional, as something the Communists would

do, something very un-American.

In addition, however, mass surveillance of an entire population cannot find

terrorists. It is a probabilistic impossibility. It cannot work.

What is the probability that people are terrorists given that NSA's mass

surveillance identifies them as terrorists? If the probability is zero

(p=0.00), then they certainly are not terrorists, and NSA was wasting

resources and damaging the lives of innocent citizens. If the probability

is one (p=1.00), then they definitely are terrorists, and NSA has saved the

day. If the probability is fifty-fifty (p=0.50), that is the same as

guessing the flip of a coin. The conditional probability that people are

terrorists given that the NSA surveillance system says they are, that had

better be very near to one (p=1.00) and very far from zero (p=0.00).

The mathematics of conditional probability were figured out by the Scottish

logician Thomas Bayes. If you Google "Bayes' Theorem ", you will get more

than a million hits. Bayes' Theorem is taught in all elementary statistics

classes. Everyone at NSA certainly knows Bayes' Theorem.

To know if mass surveillance will work, Bayes' theorem requires three

estimations:

1) The base-rate for terrorists, i.e. what proportion of the population are

terrorists.

2) The accuracy rate, i.e., the probability that real terrorists will be

identified by NSA;

3) The misidentification rate, i.e., the probability that innocent citizens

will be misidentified by NSA as terrorists.

No matter how sophisticated and super-duper are NSA's methods for

identifying terrorists, no matter how big and fast are NSA's computers,

NSA's accuracy rate will never be 100% and their misidentification rate will

never be 0%. That fact, plus the extremely low base-rate for terrorists,

means it is logically impossible for mass surveillance to be an effective

way to find te rrorists.

I will not put Bayes' computational formula here. It is available in all

elementary statistics books and is on the web should any readers be

interested. But I will compute some conditional probabilities that people

are terrorists given that NSA's system of mass surveillance identifies them

to be terrorists.

The U.S. Census shows that there are about 300 million people living in the

USA.

Suppose that there are 1,000 terrorists there as well, which is probably a

high estimate. The base-rate would be 1 terrorist per 300,000 people. In

percentages, that is .00033% which is way less than 1%. Suppose that NSA

surveillance has an accuracy rate of .40, which means that 40% of real

terrorists in the USA will be identified by NSA's monitoring of everyone's

email and phone calls. This is probably a high estimate, considering that

terrorists are doing their best to avoid detection. There is no evidence

thus far t hat NSA has been so successful at finding terrorists. And suppose

NSA's misidentification rate is .0001, which means that .01% of innocent

people will be misidentified as terrorists, at least until they are

investigated, detained, and interrogated. Note that .01% of the US

population is 30,000 people. With these suppositions, then the probability

that people are terrorists given that NSA's system of surveillance

identifies them as terrorists is only p=0.0132, which is near zero, very far

from one. Ergo, NSA's surveillance system is useless for finding

terrorists.

Suppose that NSA's system is more accurate than .40, let's say, .70, which

means that 70% of terrorists in the USA will be found by mass monitoring of

phone calls and email messages. Then, by Bayes' Theorem, the probability

that a person is a terrorist if targeted by NSA is still only p=0.0228,

which is near zero, far from one, and useless.

Suppose that NSA's system is really, really, really good, with an accuracy

rate of .90, and a misidentification rate of .00001, which means that only

3,000 innocent people are misidentified as terrorists. With these

suppositions, then the probability that people are terrorists given that

NSA's system of surveillance identifies them as terrorists is only p=0.2308,

which is far from one and well below flipping a coin. NSA's domestic

monitoring of everyone's email and phone calls is useless for finding

terrorists.

NSA knows this. Bayes' Theorem is elementary common knowledge. So, why

does NSA spy on Americans knowing it's not possible to find terrorists that

way? Mass surveillance of the entire population is logically sensible only

if there is a higher base-rate. Higher base-rates arise from two lines of

thought, neither of them very nice:

1) McCarthy-type national paranoia;

2) political espionage.

The whole NSA domestic spying program will seem to work well, will seem

logical and possible, if you are paranoid. Instead of presuming there are

1,000 terrorists in the USA, presume there are 1 million terrorists.

Americans have gone paranoid before, for example, during the McCarthyism era

of the 1950s. Imagining a million terrorists in America puts the base-rate

at .00333, and now the probability that a person is a terrorist given that

NSA's system identifies them is p=.99, which is near certainty. But only if

you are paranoid. If NSA's surveillance requires a presumption of a million

terrorists, and if in fact there are only 100 or only 10, then a lot of

innocent people are going to be misidentified and confidently mislabeled as

terrorists.

The ratio of real terrorists to innocent people in the prison camps of

Guantanamo, Abu Ghraib, and Kandahar shows that the U.S. is paranoid and is

not bothered by mistaken identifications of innocent people. The rati o of

real terrorists to innocent people on Bush's no-fly lists shows that the

Bush administration is not bothered by mistaken identifications of innocent

Americans.

Also, mass surveillance of the entire population is logically plausible if

NSA's domestic spying is not looking for terrorists, but looking for

something else, something that is not so rare as terrorists. For example,

the May 19 Fox News opinion poll of 900 registered voters found that 30%

dislike the Bush administration so much they want him impeached. If NSA

were monitoring email and phone calls to identify pro-impeachment people,

and if the accuracy rate were .90 and the error rate were .01, then the

probability that people are pro-impeachment given that NSA surveillance

system identified them as such, would be p=.98, which is coming close to

certainty (p_1.00). Mass surveillance by NSA of all Americans' phone calls

and emails would be very effective for dom estic political intelligence.

But finding a few terrorists by mass surveillance of the phone calls and

email messages of 300 million Americans is mathematically impossible, and

NSA certainly knows that.

--Floyd Rudmin is Professor of Social & Community Psychology at the

University of Tromsø in Norway. He can be reached at frudmin@psyk.uit.no