pageok
pageok
pageok
NSA data mining and the false trade-off between privacy and security:

Defenders of the NSA's known domestic surveillance programs -- listening in to some international calls and collecting records of Americans' wholly domestic phone calls -- sometimes claim that we must sacrifice a little privacy/liberty in order to gain security from future terrorist attacks. That sounds reasonable and pragmatic, as long as the magnitude of the loss of privacy/liberty is worth it in light of the magnitude of the gain in security. But in the case of the two programs revealed in the last 6 months, and especially the massive call-records programs revealed by USA Today earlier this month, the tradeoff may well be a false one. The program may well be all pain and no gain. This column by Bruce Schneier, an expert on data systems and privacy, points to a big problem with data mining of the sort the NSA is doing with Americans' telephone calls. It turns out to be a huge investment of wasted time and resources chasing rabbit trails. Writes Schneier:

Data mining works best when you're searching for a well-defined profile, a reasonable number of attacks per year, and a low cost of false alarms. Credit-card fraud is one of data mining's success stories: All credit-card companies mine their transaction databases for data for spending patterns that indicate a stolen card.

Many credit-card thieves share a pattern -- purchase expensive luxury goods, purchase things that can be easily fenced, etc. -- and data mining systems can minimize the losses in many cases by shutting down the card. In addition, the cost of false alarms is only a phone call to the cardholder asking him to verify a couple of purchases. The cardholders don't even resent these phone calls -- as long as they're infrequent -- so the cost is just a few minutes of operator time.

Terrorist plots are different; there is no well-defined profile and attacks are very rare. This means that data-mining systems won't uncover any terrorist plots until they are very accurate, and that even very accurate systems will be so flooded with false alarms that they will be useless.

Just in the United States, there are trillions of connections between people and events -- things that the data-mining system will have to "look at" -- and very few plots. This rarity makes even accurate identification systems useless.

Let's look at some numbers. We'll be optimistic -- we'll assume the system has a one in 100 false-positive rate (99 percent accurate), and a one in 1,000 false-negative rate (99.9 percent accurate). Assume 1 trillion possible indicators to sift through: that's about 10 events -- e-mails, phone calls, purchases, Web destinations, whatever -- per person in the United States per day. Also assume that 10 of them actually indicate terrorists plotting.

This unrealistically accurate system will generate 1 billion false alarms for every real terrorist plot it uncovers. Every day, the police will have to investigate 27 million potential plots in order to find the one real terrorist plot per month. Clearly ridiculous.

This isn't anything new. In statistics, it's called the "base rate fallacy," and it applies in other domains as well. And this is exactly the sort of thing we saw with the National Security Agency (NSA) eavesdropping program: The New York Times reported that the computers spat out thousands of tips per month. Every one of them turned out to be a false alarm, at enormous cost in money and civil liberties.

Finding terrorism plots is not a problem that lends itself to data mining. It's a needle-in-a-haystack problem, and throwing more hay on the pile doesn't make that problem any easier. We'd be far better off putting people in charge of investigating potential plots and letting them direct the computers, instead of putting the computers in charge and letting them decide who should be investigated.

By allowing the NSA to eavesdrop on us all, we're not trading privacy for security. We're giving up privacy without getting any security in return.

With respect to the domestic call-records program, perhaps the NSA has developed a very precise formula for pinpointing patterns of terrorist-related calls that reduce the wasted time and resources that would otherwise be expended. Perhaps there are real and verifiable success stories -- foiled plots, arrested would-be terrorists -- that have come from the NSA's activities. If so, we've seen little evidence of it, apart from the administration's unsupported assertions that these NSA programs are needed for national security. Aside from the possible unconstitutionality of one or both of the NSA programs, there's a deeper problem with the administration's position. When it comes to the loss of personal privacy and liberty, the history of the abuse of executive power and the ever-present danger of the inadvertent disclosure of Americans' personal data counsel that "Trust us" shouldn't be good enough.

Enoch:
Finding terrorism plots is not a problem that lends itself to data mining.

Flat wrong. The German authorities captured members of the Red Army Faction back in the 1970s and 1980s precisely through the application of data mining techniques, primitive though these were at the time.
5.31.2006 7:55pm
agog:
We'd be far better off putting people in charge of investigating potential plots and letting them direct the computers, instead of putting the computers in charge and letting them decide who should be investigated.

Who's to say that's not what NSA is doing? I'd imagine it's a lot easier to track terror suspects, their contacts, and the various networks to which they belong when you've got full access to a dataset of every call rather than having to track down a warrant for each new lead. I'd imagine some combination of traditional data mining techniques with inputs based on human intelligence could be quite useful when applied to this call dataset.
5.31.2006 8:09pm
agog:
Apparently, I'm doing a lot of imagining.
5.31.2006 8:11pm
David Walser:
Given that we don't know the details of what's being done, how can anyone speak authoritatively on whether or not this is a "good" use of NSA's resources? My own speculation is that, once a terrorist suspect has been identified, NSA is using its database of call records to determine what phone numbers that suspect has been calling (and which numbers have been calling the suspect). I would imagine that learning that several known terrorists have been calling the same number would be very useful in identifying new investigative leads, as would learning that the owner of a particular phone had been calling several known terrorists. Seems like common sense that this would be a very effective way of developing information about terrorists. If I had all the education and training of Mr. Schneier, I'd probably understand, as he clearly does, that what seems like common sense to the ignorant is truly the height of folly to the educated -- but I doubt it.

Whether this is or is not a good use of NSA's resources, of course, is of little import to the constitutional question.
5.31.2006 8:47pm
Medis:
I have no independent ability to analyze the likely cost-benefit outcome of these NSA programs. I am deeply worried, however, about the ability of this Administration to reliably carry out such an analysis itself.
5.31.2006 9:02pm
David Walser:
Medis: "I have no independent ability to analyze the likely cost-benefit outcome of these NSA programs. I am deeply worried, however, about the ability of this Administration to reliably carry out such an analysis itself."

So? Are you saying that this Administration, because you find it lacking in competency, should NOT have the the same authority in this area as past and future (competent) Administrations? Or are you saying that NO Administration should be trusted with this authority? The first position is an argument for winning the next election. The second is an argument for appropriate legislation (or constitutional amendment). Neither position seems applicable to whether the current Administration has the authority to do what it's been doing.
5.31.2006 9:37pm
Christopher Cooke (mail):
I read a similar criticism by a mathematician in a NY Times column a few weeks ago. He essentially pointed out what he called the "Six Degrees of Kevin Bacon" problem, which is that everyone is connected to everyone else, ultimately, so you come up with many useless "connections" through this type of data mining.
5.31.2006 9:40pm
Classmate-Wearing-Yarmulka (www):
Since no one knows exactly what the NSA is actually doing with all the information it has, any attempts to figure out the of it is silly at best.
5.31.2006 9:44pm
Adam S (mail) (www):
It's not precisely the same, but I've blogged about "money laundering" and data mining in Fincen Effectiveness

There's a huge amount of effort to track down a very small sliver of the money that drug dealers move around.
5.31.2006 9:51pm
Medis:
David,

I took Dale to be posting about a policy issue--whether or not the NSA programs have resulted in a sufficient "gain" in national security to justify the associated "loss" of privacy/liberty. That issue can be discussed without actually getting into the issues of whether the Administration does have the legal authority, or should have the legal authority, to institute such programs. In other words, one can look at this from a policy perspective without considering the associated legal issues.

That said, I would in general argue that no Administration should be trusted to properly weigh national security interests against individual privacy/liberty interests behind closed doors, and with no substantive participation by the other branches of government. And in a nutshell, I think that is true because history shows that government officials tend to overestimate the gains to security and underestimate the losses to privacy/liberty in such situations, perhaps because of the very common tendency of government officials to rationalize increasing their own power whenever they can.
5.31.2006 9:56pm
Richard Aubrey (mail):
The government had enough information on 26.5 million veterans that a stolen laptop bids fair to screw a bunch of people. But nobody was concerned that the feds had that much info on each individual, which seems to be a good deal more than the NSA has or is looking for.

I recently got a mortgage offer. Some outfit in Texas knew my starting amount, my monthly payment, and my balance. As a bit of roccoco dressing, they knew we'd been paying ahead, so the balance we owed was somewhat less than what could be deduced from looking at amount, time, and interest and checking an amortization table.
I have no idea what else on me is floating around.
The IRS has a lot on me.

What is it about this phone number thing that seems to be so much more important?

Don't bother to hyperventilate about "liberties". If info equals loss of liberties, mine are already pretty lost to any itinerant mortgage broker, IRS functionary, or whoever bought a stolen laptop. If less info means less loss of freedom, then the NSA isn't much of a threat.

And to all of you itching to sneer by quoting "Don't bother to hyperventilate about "'liberties'", out of context, you're bus tid in advance.
5.31.2006 10:30pm
EricK:
I would like for anyone to tell me, how anyones liberty is lost by this program.
5.31.2006 10:47pm
frankcross (mail):
The data mining itself doesn't seem to infringe on liberty, but Dale's point is a good one that is often overlooked.

The infringement would come because the data mining would flag a person's calls and the government would then listen to the calls. But because of the absolute number of errors (even with a good error rate), the person would be innocent. A 1 in 1000 chance of guilt doesn't sound like probable cause.
5.31.2006 11:05pm
Richard Aubrey (mail):
frankcross. Do you have information this could be done without the usual warrant?
5.31.2006 11:07pm
Medis:
EricK,

Well, let me ask a hypothetical. Suppose the government wanted to start charging U.S. citizens $100 every time they attended church. Would any liberty be lost by such a program?
5.31.2006 11:14pm
John (mail):
Here's the dopey part: "By allowing the NSA to eavesdrop on us all..." That's not what's happening. The loss of privacy is de minimus.

As to the "gains" from the programs, we know very little of what is going on. The people who work as the specialists at NSA are VERY smart mathematicians. Including statisticians. They may not be having a lot of success (who knows, after all?), but they apparently do not think their work is useless, and they would know.
5.31.2006 11:40pm
Medis:
John,

Of course, we don't know if the decision to proceed with this program was being driven from the bottom up in that sense--and all too many crucial decisions by this Administration have worked from the top down.
5.31.2006 11:43pm
Christopher Cooke (mail):
To Richard Aubrey: See last week's New Yorker article by Seymour Hersh (Talk of the Town section), in which he quotes NSA sources stating that the NSA was monitoring the contents of individual phone calls in the US (not US to foreign locations, or foreign to foreign calls) without obtaining FISA or Wiretap statute warrants, as a follow-up to the NSA's data mining project. There is no "liberty" lost by this program, just privacy.
5.31.2006 11:46pm
Enoch:
I think the evidence accessible through Google contradicts your claim Do you have any evidence to support it?

Google harder. I can find more if you like.

How Terror Groups End, by Chris Harmon, Ph.D., Kim T. Adamson Chair of Insurgency &Terrorism, Marine Corps University

Effective leadership employing the full range of national power (from political and military through economic and informational) has led to the demise of terror groups. For example, the German government's focused effort and discipline wore out the Red Army Faction (aka the Baader-Meinhof group) in the late 1970s. The Germans employed new methods of policing, including an early form of data-mining and a new counter-terrorism unit (created after the 1972 Olympics attack by Palestinian radicals), to bring down the group.

Mission impossible?
J. Kumagai, IEEE Spectrum, 4 April 2003

In the late 1960s and 1970s, the Red Army Faction (RAF) and
other militant groups unleashed a stream of bombings, robberies, kidnappings, and murders throughout Germany. Faced with this growing threat, Horst Herold, head of Germany's Bundeskriminalamt (the BKA, or federal police), had a bold plan: to create a computer network to mine data from real-estate agencies, utility companies, and other sources, in hopes of pinpointing the terrorists' whereabouts.
...
Profile in hand, the police contacted electricity companies,
to find out which apartments used no or little electricity, and apartment complexes, to find out which people paid in cash; they also combed through household registrations (German citizens are required to register with the state). "The results were all merged, and in the end, they found one flat which fit absolutely this profile," Garstka says. Police put the apartment under surveillance and soon nabbed RAF member Rolf Heissler.
5.31.2006 11:53pm
anonymous (mail):
"Perhaps there are real and verifiable success stories -- foiled plots, arrested would-be terrorists -- that have come from the NSA's activities. If so, we've seen little evidence of it"...

Nor will you, hopefully. That's why they call it "spying."

Geez. Isn't there a minimal level of intelligence necessary to blog on this site?

Look, if you don't feel we're being protected by our current government (remind me why we have not been attacked since 9-11, why don't you), then campaign to throw the current government out.

But don't be a simpleton and demand that you be presented with the results of our efforts in secrecy.

All pain, no gain? Why do I seriously doubt that the numeric integer that describes the pain you've been dealt by the NSA programs is precisely, arithmetically equal to ... wait for it ... zero.
6.1.2006 12:13am
anonymous (mail):
Make that "not seriously doubt"
6.1.2006 12:14am
JHW (mail):
I can't imagine randomly looking through trillions of phone connections to find patterns. However, I can imagine that having a dataset of number-number connections would be very useful in determining whether there are interconnections between calls from known bad-guys from outside the country to a phone number in the country and the numbers that that number calls. Matching these connections with other connections from other known bad-guys could reveal a set of contacts. Some might be calls for pizza delivery but the numbers in common, especially over time, and from different sets of known badguys, could reveal a group with common interests that could lead to a warrent for the common connections.
6.1.2006 12:17am
Richard Aubrey (mail):
I presume Hersch hasn't identified his sources?

I don't have easy access to the New Yorker--their idea of fiction is extremely strange--so I will ask what, if anything, has been done about warrantless wiretapping? Does it fall within a category which would normally not require a warrant? Has anybody gone to the DOJ? Their response?

Oh, yeah. Is it true?
6.1.2006 12:21am
Questioner:
Bruce Schneier:

"Data mining works best when you're searching for a well-defined profile, a reasonable number of attacks per year, and a low cost of false alarms. Credit-card fraud is one of data mining's success stories: All credit-card companies mine their transaction databases for data for spending patterns that indicate a stolen card.

Many credit-card thieves share a pattern -- purchase expensive luxury goods, purchase things that can be easily fenced, etc. -- and data mining systems can minimize the losses in many cases by shutting down the card. In addition, the cost of false alarms is only a phone call to the cardholder asking him to verify a couple of purchases. The cardholders don't even resent these phone calls -- as long as they're infrequent -- so the cost is just a few minutes of operator time.

Terrorist plots are different; there is no well-defined profile and attacks are very rare. This means that data-mining systems won't uncover any terrorist plots until they are very accurate, and that even very accurate systems will be so flooded with false alarms that they will be useless.

Just in the United States, there are trillions of connections between people and events -- things that the data-mining system will have to "look at" -- and very few plots. This rarity makes even accurate identification systems useless."

Wow! You mean a government with unlimited powers to tax and spend other peoples' money is willing to spend large sums on dubious projects with incredibly low payoffs rather than engage in rational cost-benefit assessments as profit-and-loss companies do? Who'd a thought?

If cardholders found their prices went up as a result of poorly spent "identity theft protection" offered by some companies, they'd switch credit cards, a choice less easily made by our government's "customers".

The government, eager to tell unsophisticated citizens it is doing SOMETHING to protect them from terrorism force them to pay for the illusion of safety. THIS is what Americans have given up liberty and privacy for. Ben Franklin would be so proud...not.
6.1.2006 12:22am
Wintermute (mail) (www):
Troll city. Sieg heil!
6.1.2006 12:23am
Grover Gardner (mail):
Enoch--

This doesn't sound to me so much like "data mining" as good old fashioned detective work. They started with a very clear idea of who and what they were looking for ("Profile in hand..."), as opposed to randomly sifting through data searching for apparent patterns generated by people they didn't know. I really wonder if your comparison is valid.

Richard Aubrey--

Of course your mortgage is a matter of public record, as is how much you've paid on it. This all on your credit record. But a mortgage company doesn't have the right to eavesdrop on your private phone calls to determine if you're really credit-worthy. They can only judge that based on publicly available information. Nor could an insurance company deny you coverage based on a call to your brother during which you complained of persistent headaches and dizzy spells.
6.1.2006 12:31am
coggieguy (mail):
The point is not to find th eneedle in the haystack in ONE day. run this prodcdure for 30, 60, 90 or 180 days and find the hits that continually coem through.

Wow - Its been a while since I did serious statistics, but there are lots of techniques for finding real data in the noise.

And another thing - Schneier should at least call false positives and false negatives by the common names for these errors - alpha and beta. Anyone with a passing acquaintance with statistics knows about alpha error - and you develop methods to test these quickly. (ANY number crunching program - R&D, spying, census adjustment etc. knows how to do this) It is the beta errors that cause me to lose sleep at night - finding a terror cell and deciding it isn't real. This is the dynamic in any statistical driven system - apha versus beta. And for extra sleep deprivation - if you cut the alpha (say civil libertarians put restrictions on false positives) the beta error rate goes UP. (the opposite is also true) Restricting the false positives increases the chances we miss the bad guys.
6.1.2006 12:38am
Grover Gardner (mail):
"...so I will ask what, if anything, has been done about warrantless wiretapping? Does it fall within a category which would normally not require a warrant? Has anybody gone to the DOJ? Their response?"

Just because you don't get the New Yorker doesn't mean you have to be completely oblivious.
6.1.2006 12:41am
Grover Gardner (mail):
Enoch,

Reading the article you quoted, I see about three paragraphs from your own quote a very negative assessment of modern data mining techniques. I think your point is somewhat contradicted by this--if we are to read the article as having any sort of authority.
6.1.2006 12:58am
Lev:

perhaps the NSA has developed a very precise formula for pinpointing patterns of terrorist-related calls that reduce the wasted time and resources that would otherwise be expended. Perhaps there are real and verifiable success stories -- foiled plots, arrested would-be terrorists -- that have come from the NSA's activities. If so, we've seen little evidence of it,


What do you expect? That the formulas and results should be published in the newspaper? What kind of idiot are you?
6.1.2006 1:05am
David Gaw (mail) (www):
If it is reasonable to appeal to Schneier's authority as a security expert, surely we should also consider that the agency that devised the surveilence programs in question is staffed by people who are experts in data surveillence and who, presumably unlike Schnier, are in a position to be well-informed about the details.

It could be that Schneier's assessment of the program is superior to the NSA's despite his lack of information... or it could be that his well-publicized dislike of the programs in question, and his feeling that "we're giving up privacy without getting any security in return" is coloring his analysis.

Either way, in the absence of evidence he has the inside scoop on how the programs really work, for him to dream up a scenario that may have no relationship to how the NSA is using the data it collects, then invent statistics related to his imaginary program comes across as straw man demagogery of the worst sort.

If Bruce is going to fabricate a scenario that depicts the NSA program in the way he finds expedious, can we all join in? Here's an alternative scenario: let's suppose the NSA uses interceptions of calls between known terror subjects outside the US to identify phone numbers of interest in the United States. Ownership of these domestic phone numbers become a well-defined profile of exactly the sort that Schnier agrees works well with data mining. The NSA can then use its database of call records to identify patterns and relationships between people who are chatting with known terrorists and other phone numbers, then use that information to identify patterns and develop possible surveilence targets within the US. I would also think such a call database could be used after a successful terrorist attack to backtrack and find and apprehend conspirators. People might or might not like the approach, but it would seem to have the potential to provide a lot more than "no return."
6.1.2006 1:15am
Christopher Cooke (mail):

so I will ask what, if anything, has been done about warrantless wiretapping? Does it fall within a category which would normally not require a warrant? Has anybody gone to the DOJ? Their response?

Oh, yeah. Is it true?

To take your last question, of course, I have no idea if his story is correct, but Hersh did break the Abu Ghraib story as well as the My Lai story (many years ago) and appears to have excellent sources, based on that track record.

Regarding warrantless wiretaps, FISA states that all wiretaps have to be either authorized under FISA or the Wiretap statute, there is no other way (see Professor Kerr's posts on this). So, putting aside the "King can do no wrong" defenses of Bush's purported absolute authority to do whatever he wants, Congress be damned, I would say that all warrantless wiretapping in the US, of calls between persons in the US, is likely illegal. Has anything been done? No, the Republicans who control Congress seem to be more outraged by the fact that a soon-to-be- convicted felon member of Congress' office was searched by the FBI, than by these apparent transgressions.
6.1.2006 1:26am
Thomasly (mail):
It's my understanding--and I can't find the source, so if you can help or contradict, please do--that the NSA believes it does have some profiles. There were, after all, 19 hijackers living here but contacting others abroad. There may well have been similar profiles developed from other al Qaeda attacks. If that's the case, it seems to me much of the complaint simply goes away.
6.1.2006 1:26am
LTEC (mail) (www):
Schneier says: "Terrorist plots are different; there is no well-defined profile ..." I wonder how he came to believe himself to be an expert on terrorism. In fact, such profiles mean that the government should be able to get by without unrestricted access to all phone call records.

By the way, I wish our government would start to treat street gang crime seriously, even if it doesn't qualify as "terrorism" due to a lack of political motive. No "profiling" is necessary and no list of phone records is needed. The police know who most of the criminals are (it's hard not to know) and where they live, and it is trivial to catch them committing crimes and amass evidence against them and convict them. Imagine that Al Queda members wore jackets that said "Al Queda".

And as long as I'm rambling, the shocking thing about the above post on the lack of room in L.A. jails is the fact that Americans are not willing to pay the cost of the criminal justice system that they want and need, anymore than they are willing to have our government study and infiltrate Islamist groups (or at least the seeming worst of them). But they are willing to support "data mining" no matter what it means, as long as everybody's privacy is being equally violated.
6.1.2006 1:27am
Bruce Hayden (mail) (www):
I found it enlightening that the author jumped from data mining to evesdropping at the end, when these appear to be completely different programs. If he truly believes his billion or so to one false positive rate, then the NSA is evesdropping on every conversation in this country. But, of course, that is ridiculous. I still haven't seen anything the least bit authoritative that the NSA is intercepting any strictly domestic calls without at least FISA warrants.
6.1.2006 1:36am
Medis:
David Gaw,

Again, it seems to me you are relying on the assumption that it is the experts in the NSA who are driving the crucial decisions. I do not think that is a safe assumption.
6.1.2006 2:21am
Questioner:
Bruce Hayden: "I still haven't seen anything the least bit authoritative that the NSA is intercepting any strictly domestic calls without at least FISA warrants."

David Friedman (Santa Clara School of Law) suggested months ago on his blog that a straight-forward explanation for why Bush's branch wouldn't try to get FISA warrants, even after the fact, even though FISA authorizes over 99% of requests, is that they ARE listening in on virtually ever phone conversation in America, and both doesn't want that known and know even FISA won't authorize that.

But David's just an economist, thinking in terms of empirical observations and reflections on incentives...
6.1.2006 4:55am
o' connuh j.:
Again, it seems to me you are relying on the assumption that it is the experts in the NSA who are driving the crucial decisions. I do not think that is a safe assumption.


Safer than assuming the contrary, for which you have no evidence whatsoever.
6.1.2006 6:14am
Jeek:
This doesn't sound to me so much like "data mining" as good old fashioned detective work. They started with a very clear idea of who and what they were looking for ("Profile in hand..."), as opposed to randomly sifting through data searching for apparent patterns generated by people they didn't know. I really wonder if your comparison is valid.

I question your assumption that the NSA does not have a very clear idea of who and what they are looking for, and are "randomly sifting through data searching for apparent patterns generated by people they don't know." The idea that the NSA is starting with a "blank slate" seems ridiculous on the face of it. They should certainly have enough information about these bad guys to construct a profile that is at least as useful as the primitive-sounding profile the Germans created to find RAF members, if not more.

David Gaw, your remarks are right on target.
6.1.2006 9:09am
Richard Aubrey (mail):
Grover. My mortgage information is a matter of public record. So? My point is my privacy is breached by a number of institutions. To declare that, since my mortgage info is public record I'm not supposed to worry is nuts.
The point is, with all this crap floating around out there on me, and with the charitable presumption that other stuff that's not supposed to be floating around--like my IRS stuff--isn't, why is the HSA's work the major threat?

As to life insurance. You sign an authorization allowing any medical facility the company wishes to contact to tell them anything they want to know. It's entirely voluntary. Maybe you don't need the life insurance? You give it up or you don't even get an application processed.
6.1.2006 9:09am
SLS 1L:
Don't forget that the dataset would be extremely, extremely useful for keeping tabs on the administration's enemies and finding anonymous sources.
6.1.2006 9:48am
johnt (mail):
Somewhere in that universe of phone calls my number must be floating around, and more than once, and for a few years. Despite a vigorous retrospective of my recent years still I fail to encounter any loss of privacy or civil liberties. Limited though it is, none of my friends have lost privacy/liberty, or if they have they're not telling. And they're on the phone more than I am.
Feverishly I check the papers for tales of unfortunates who, due to NSA, have been deprived of privacy/liberty. Happily I come up short despite the anticipation of the papers of a fascist reign of terror.
Am I a pollyanna? Much as I wish to be terrorized I am starting to suspect that this is all b______t, even though I would prefer to be terrorized and therfore indignant.
Now however I must temporarily turn my attention to the Internal Revenue Service where privacy/liberty are never issues.
6.1.2006 10:03am
Ken B:
Isn't the argument of the quoted article basically "I don't know how to do it, so it can't be done"?
6.1.2006 10:37am
Medis:
o'connuh,

I'm not actually assuming anything in particular--I'm just pointing out that arguments to the effect that these programs must be a good idea because experts in the NSA have concluded that they are a good idea depend on unwarranted assumptions about the Administration's decision-making process, and people need not reach the opposite assumption to note the weakness of these arguments.

That said, I would point you to the many cases in which we have in fact gotten a window into the Administration's decision-making process, and we have seen that all too often, crucial strategic and tactical decisions are made at a "high-level" without consultation with the relevant experts in the relevant departments and agencies. Subsequently, the role of those experts has been to justify the decisions that were already made at a "high-level", and experts who have failed to understand and play this role have been ignored and ultimately dismissed. And, unfortunately, many of these decisions have led to counterproductive and even disasterous results.

So, the fact that we have observed this general pattern of decision-making in other cases does in fact provide grounds to be concerned that this pattern is repeating itself in these cases as well.
6.1.2006 10:39am
frankcross (mail):
Data mining can be a very useful technique that is perfectly appropriate to fighting terrorism.

However, in other circumstances it can be a useless technique. The point of the article is that this is precisely the circumstance where it is useless -- you have an enormous population being investigated and an extremely small number of true positives. In that case, you are going to get very large numbers of false positives even with an extremely accurate test. That's not inside information, that's just mathematics.

The issue is what is then done. If the very high false positive rate means that the government won't be able to surveil, then the program is useless. If it is considered good enough for surveillance, then you have liberty concerns.
6.1.2006 10:58am
Zach (mail):
I'm unimpressed with Schneier's train of reasoning here. It seems to be that the largest employer of mathematicians in the world is devoting lots of resources to a project implemented (in Schneier's imagination) in a way which any mathematician would predict failure. Isn't it simpler to believe that the NSA is using the information in a way which they believe has validity?

The reasoning Schneier uses here is not very difficult or esoteric, and there's absolutely no reason to think that he's thought of something that anybody with a calculator and some common sense wouldn't think of in ten minutes. I'm open to the idea that large organizations can do things more stupidly than their individual members, but this seems like walking up to an accounting firm and pointing out that money made next year is less valuable than money made this year. The organization is full of people who know that just as well as you do.

Just to throw out one suggestion that would be workable: Suppose the NSA is keeping a database of all known terrorist-linked phone numbers and their calling records (international numbers and calling records). If we suppose that terrorist cells keep in regular contact both with each other and with the home organization, we can use a database of domestic calling records (assuming we have it) to find out what domestic numbers are most closely linked to the known terrorist-linked phone numbers. This suggestion is almost as simple as Schneier's picture, but it would be workable. There are things terrorists could do to decrease the effectiveness, like switching phones regularly or using one phone to receive calls and one to make them, but those also increase the difficulty for terrorists to keep in contact with one another. I suspect that tracking down "dirty" numbers would pay off pretty well in practice.

For what it's worth, somebody with the 9/11 Report in front of them might try to look up what Atta's calling patterns were, assuming that that information made it into the report. I recall that Atta actually travelled a fair amount, so I wouldn't be surprised if he also made a lot of phone calls setting up meetings, arranging for funds, etc.
6.1.2006 11:05am
A.S.:
Let's look at some numbers. We'll be optimistic -- we'll assume the system has a one in 100 false-positive rate (99 percent accurate), and a one in 1,000 false-negative rate (99.9 percent accurate).

How does the author of this article know enough about the program to determine whether those numbers are "optimistic"?

Does he present any evidence to support those numbers? There's none that I can see. It seems to me that the entire article is bullsh*t. The author has no idea what the NSA is even doing in the data mining program (nor do any of the rest of us, based on publicly availble information), much less less how accurate the program is.

Let's be optimistic: we'll assume the system has a one in 1,000,000,000 false-positive rate, and a one in 100,000,000,000 false-negative rate. Hey, I've got every much a right to pull numbers out of my *ss at the author of the article. So let's run the numbers on the program now.

Assume 1 million possible indicators to sift through: why do we assume that the program collects everything it is possible to cellect, or that it "sifts through" everything it collects? Also assume that 1,000 of them actually indicate terrorists plotting.

NOW, somebody tell me how many terrorist plots the program has uncovered.
6.1.2006 11:05am
Freder Frederson (mail):
Safer than assuming the contrary, for which you have no evidence whatsoever.

It is becoming more and more obvious that this administration is digging into the private lives of U.S. residents and is willing to lie about it and use the full force of the law to conceal their activities. Yesterday it was revealed, that after the DOJ assured the Congress and the American People that the library snooping provisions of the Patriot Act had never been used, they actually had, but the librarians who had been served with orders to turn over records of individual library patrons were under court order not to discuss it. Only in the past few days has the order been lifted and the librarians can reveal the Administration and even the AG for what they truly are--liars.
6.1.2006 11:08am
Freder Frederson (mail):
The stated reason for collecting this data doesn't even make sense. If you were indeed only tracking the phone calls of AQ operatives, you would know at least one phone number. That would give you the "root" of a call tree. You could then start with that number and analyze all the calls related to that number.

We are not talking about just phone numbers here, we are talking about call patterns; who is called, time and duration of calls. To say the information is somehow is anonymous is ridiculous. The NSA can easily reconnect the numbers with customer information (as any of us could do). I suspect that the first thing the NSA does is match up phone numbers with Moslem organizations and last names and looks for frequent calls to Mosques or suspected terrorist linked Moslem charitable organizations.
6.1.2006 11:19am
Paul Gowder (mail):
Ssshhh! Citizens aren't supposed to know about Bayes' rule! Bruce Schneier must be in league with the terrorists. Didn't Arabs invent math?
6.1.2006 11:23am
Medis:
Zach,

I think you understate the problem when you describe it as "large organizations can do things more stupidly than their individual members." Here we are dealing with a very specific kind of organization: a governmental agency which ultimately is controlled by political actors who are not in fact part of that agency. In such cases in general, it is not uncommon for the political actors to order the agency to do things which many experts in the agency would claim is a bad idea. And we know that the particular political actors in question in this case are particularly likely to do such a thing.

In other words, a better analogy would be asking if the White House OMB would be likely to use accounting methods that ordinary accountants would say are improper because the political actors in the White House wanted the OMB to do so. And is that really an unrealistic scenario? Because actually, it happens all the time.
6.1.2006 11:23am
Freder Frederson (mail):
Of course real terrorists would just buy a DVD of HBO's "The Wire" and quickly learn how to circumvent all the problems associated with phone tapping and NSA surveillance problems. And they wouldn't have the problem of lazy drug dealers with pain-in-the-ass girlfriends who fail to follow instructions and buy all their throw-away cell phones at one store instead of limiting purchases to two at a time.
6.1.2006 11:24am
Bruce Hayden (mail) (www):
Questioner

Or, maybe, just maybe, the Administration is telling the truth that it just takes too long to get FISA warrants when they need them. Several people who have worked in the area of getting FISA warrants have stated that normal FISA warrants take several weeks to a month to get all the paper work together. The AG has repeated stated that even the Emergency Orders provision doesn't help much, since, according to him, the paper work isn't reduced, just the time frame.

The other thing that they have talked about is that they often don't have much time here, in the order of minutes, or at worst, hours, as opposed to the days (under the Emergency Orders provision) or weeks or more required, when something is breaking. We do know, from other sources, that one of the things that the NSA is doing is having the troops feed call history and phone book phone numbers from phones seized in raids in Iraq and Afganistan to them in real time. They then have hours at most before these numbers go dead, to trace the calls from or to those numbers. This may be a separate program, but to me, it makes more sense that it is tied to their international surveilance program than the paranoid musing of a law prof that you cite (ok, I admit to being a little harsh there - but he doesn't seem to have much to base his suggestion on).
6.1.2006 11:25am
Freder Frederson (mail):
The other thing that they have talked about is that they often don't have much time here, in the order of minutes, or at worst, hours, as opposed to the days (under the Emergency Orders provision) or weeks or more required, when something is breaking. We do know, from other sources, that one of the things that the NSA is doing is having the troops feed call history and phone book phone numbers from phones seized in raids in Iraq and Afganistan to them in real time.

I call bullshit on this. The Administration can't even lie convincingly. It is amazing how many people buy their obvious bullshit. But I guess they don't have to try too hard when people and the press will accept explanation without critical analysis.

First off, I seriously doubt that U.S. troops in either Afghanistan or especially Iraq are finding any phones with U.S. numbers on them, and if they are, it is an extraordinarily rare occurance. First off, I assume that AQ and the other terrorists are smart enough to figure out that since we rebuilt Iraq's and Afghanistan's infrastructure and we control the communications satellites and transcontinental communications landlines into the U.S. that we are monitoring every call that goes in and out of those countries and probably most internal ones too. Also, if I were running a terrorist cell, my first order would be to clear the memory after every call and to destroy all cell phones if there was any possibility of capture. Finally, why on earth would the terrorists in Iraq be calling the U.S. when they have 135,000 targets right in their back yard?

FISA allows for immediate tapping with a 72 hours to get a warrant after the tap is placed. If the President thinks this is unworkable, then his remedy is to ask the Congress to change the law to make it easier for him to comply, not just say it doesn't apply.
6.1.2006 11:46am
A.S.:
FISA allows for immediate tapping with a 72 hours to get a warrant after the tap is placed.

This is, of course, a lie.

FISA allows for surveillance for 72 hours after the Attorney General determines that there is enough evidence to obtain a FISA warrant. Not "immediate tapping".
6.1.2006 11:52am
Freder Frederson (mail):
FISA allows for surveillance for 72 hours after the Attorney General determines that there is enough evidence to obtain a FISA warrant. Not "immediate tapping".

Oh yeah, Gonzales might determine there isn't enough evidence to obtain a FISA warrant. This is the man who never presented Bush with any mitigating evidence against the application of the death penalty even when such evidence was abundantly available. He is nothing but a rubber stamp for anything the Administration wants to do. He thinks torture is a good idea and portions of the Geneva Convention are "quaint".
6.1.2006 11:58am
Jeek:
First off, I assume that AQ and the other terrorists are smart enough to figure out that since we rebuilt Iraq's and Afghanistan's infrastructure and we control the communications satellites and transcontinental communications landlines into the U.S. that we are monitoring every call that goes in and out of those countries and probably most internal ones too. Also, if I were running a terrorist cell, my first order would be to clear the memory after every call and to destroy all cell phones if there was any possibility of capture.

So it's not even worth trying to catch any stupid, lazy, or unlucky terrorists?

If you read anything about Allied efforts to break the Enigma code in WW2, you will see that the Allies enjoyed a great many successes as a result of enemy operator error, i.e. there were stupid, lazy, and undisciplined German operators who compromised security. And this happened despite the fact that the Germans were "smart enough" to assume the Allies were trying to break their codes, and took steps to prevent this (fortunately these were insufficient). Moreover, despite orders to destroy codebooks and Enigma machines if there was any possibility of capture, the Germans did not always manage to do so.

In short, even though we must assume that Al Qaeda is "smart enough" to realize that we're listening, and probably adopted measures to frustrate us, that does not mean that these measures will automatically succeed and therefore it's not even worth trying.
6.1.2006 12:22pm
frankcross (mail):

Just to throw out one suggestion that would be workable: Suppose the NSA is keeping a database of all known terrorist-linked phone numbers and their calling records (international numbers and calling records). If we suppose that terrorist cells keep in regular contact both with each other and with the home organization, we can use a database of domestic calling records (assuming we have it) to find out what domestic numbers are most closely linked to the known terrorist-linked phone numbers. This suggestion is almost as simple as Schneier's picture, but it would be workable. There are things terrorists could do to decrease the effectiveness, like switching phones regularly or using one phone to receive calls and one to make them, but those also increase the difficulty for terrorists to keep in contact with one another. I suspect that tracking down "dirty" numbers would pay off pretty well in practice.


This sounds like a good idea but it is not data mining and it does not require a database of all phone calls made. It is a good idea but not a justification, if the NSA is engaged in data mining on a base of all calls from at least some providers.
6.1.2006 12:24pm
Medis:
Of course, the serious point (as opposed to the unserious paperwork point) is that if the evidence is not in fact sufficient when the AG starts the surveillance, the FISA court will eventually find that out when the application is submitted. One of the oddities in the Administration's position is that they have in fact effectively claimed that there is enough evidence at the time the surveillance starts to support getting a FISA order. But they appear reluctant to allow the FISA court to review that claim after-the-fact in certain cases.
6.1.2006 12:28pm
Freder Frederson (mail):
So it's not even worth trying to catch any stupid, lazy, or unlucky terrorists?

Where did I say that? I said it is probably an extraordinarily rare occurance. AG Gonzales is not being woken up every five minutes with another list of 100 phone numbers in the U.S. that have been captured from terrorists in Iraq or Afghanistan that he has to decide whether there is sufficient evidence for a FISA warrant. I would imagine the flow of U.S. numbers is more like a trickle, or an extremely rare occurance. This country is simply not crawling with AQ operatives, as much as the Administration would like to paint the opposite picture to keep us scared in order to maintain their power (because that is the only thing that keeps the American people from realizing how truly dreadful this Administration is).
6.1.2006 12:33pm
EricK:
First off, I seriously doubt that U.S. troops in either Afghanistan or especially Iraq are finding any phones with U.S. numbers on them, and if they are, it is an extraordinarily rare occurance. First off, I assume that AQ and the other terrorists are smart enough to figure out that since we rebuilt Iraq's and Afghanistan's infrastructure and we control the communications satellites and transcontinental communications landlines into the U.S. that we are monitoring every call that goes in and out of those countries and probably most internal ones too. Also, if I were running a terrorist cell, my first order would be to clear the memory after every call and to destroy all cell phones if there was any possibility of capture. Finally, why on earth would the terrorists in Iraq be calling the U.S. when they have 135,000 targets right in their back yard?



They are finding them. Also it is a little hard to clear the memory of a phone when you are dead. Not to mention that even if you erase the memory you still can retrieve that information. Do you honestly think that terrorists are not in communication with each other?
6.1.2006 12:42pm
Freder Frederson (mail):
They are finding them.

Says who, the administration? Well, where are all these terrorists in the U.S. that the ones in Afghanistan and Iraq are calling? And I'm not talking about the poor saps who play paint ball or high school students who get tortured by the Saudis into "admitting" they were plotting to kill the president. Our courts, or at least Gitmo and the secret prisons should be full of them.

All we have had in the last four and a half years is a few trumped up charges with pathetic and laughable evidence based on obviously coerced confessions using threats of Gitmo and imprisonment without trial or access to lawyers. Makes you proud to be an American.
6.1.2006 1:03pm
Philip F. Lee (mail):
The comments here appear to miss the boat by arguing about the unknown levels of performance of the data mining system.

I would have thought the concerns for liberty to be more along the lines that notwithstanding any real claims of performance, what are the protections against abuse of the system?

Abuse should be a concern even if you believe the present adminstration is headed by a man with the moral recitude and the wisdom of Saint Augustine. What happens when we get a Richard Nixon in the office of President (or, if you don't think he was so bad, use Huey Long)?

I would really like to hear what provisions in the law prevent abuse: how would abuse be detected and how would it be stopped?
6.1.2006 1:55pm
Greg D (mail):
Bruce Schneier is an excellent source for information about cryptography. He's a prety good source for information about security in general.

However, he's had an anti-government bug up his butt about the War against the Terrorists from day one, and it's led him to say some pretty stupid things about it. You would do well not to unquestioningly rely on him just because he tells you what you want to hear. Esp. since his rant includes such obviously false to fact statements as "By allowing the NSA to eavesdrop on us all". Niether the phone call record data mining nor the "listen in on people who called or got calls for Al Qaeda" do that.

So either he's a sleazy liar, or an idiot who's pontificating about things he doesn't understand. Either way, not worth listening to.
6.1.2006 1:55pm
Mr. X (www):
"As to the "gains" from the programs, we know very little of what is going on. The people who work as the specialists at NSA are VERY smart mathematicians. Including statisticians. They may not be having a lot of success (who knows, after all?), but they apparently do not think their work is useless, and they would know."

Right. Let's ask people whether or not they think their career is useless. I'm sure we'll get an unbiased and accurate answer and a much better one than we could ever get from an neutral and independent observer.
6.1.2006 2:03pm
A.S.:
Right. Let's ask people whether or not they think their career is useless. I'm sure we'll get an unbiased and accurate answer and a much better one than we could ever get from an neutral and independent observer.

Agreed. This is always what I've said about people getting grants to research global warming. Of COURSE they are going to find evidence to support global warming! What are they going to do, find evidence that contradicts their source of funding???
6.1.2006 2:08pm
Fub:
Freder Frederson wrote:
...This country is simply not crawling with AQ operatives, as much as the Administration would like to paint the opposite picture to keep us scared in order to maintain their power (because that is the only thing that keeps the American people from realizing how truly dreadful this Administration is).
That assertion may be true, but I think it is a weaker reason to object to warrantless domestic spying, unless one expects to be dead in 2008. The stronger reason is that even if the current administration is pristinely innocent of any abuse, the next administration may not be. Nor the next after that.

Both major political parties agree that the "war on terrorism" has no definable end. The precedent for warrantless domestic spying on all citizens is now being set. At what point will "we have always been at war with Eastasia" be a statement of widely accepted background fact that will justify unlimited bureaucratic mission creep?

Republicans may rue the day that they gave government powers that they would rather Democrats not have.
6.1.2006 2:30pm
eddie (mail):
I wholeheartedly agree that the liberty/privacy issue is inextricably connected to the likelihood of abuse.

And notwithstanding any secrecy that must be maintained, I find it very troubling that individuals are not concerned about the lack of oversight in this whole procedure. I am not convinced, nor do I trust anyone who might try to convince me, that this method, its application, and the legal arguments used to back it up, are not merely a ploy to enlarge the power of law enforcement in this country and finally "get rid of the coddling of criminals".

The burden of proof should not be on us to prove what liberty or privacy interest is being violated. It seems to me that when the entire population of a "free" country is subjected to wholesale scrutiny, it is apparent that the government has crossed a line and is not a government for the people.

That "data mining" is perhaps not the same as a search is arguable, but the data itself, the large cache of information just waiting to be used, is problematic. There is also the issue of whether the government of the people should be keeping profiles of all of its citizens.

And of course where does this stop? It seems that the proponents of this sort of investigation would have no problem if cameras were mounted in all public spaces and such information was regularly "mined". Why not require the registration of every individual and regular terrorist checkups by way of questionairres. I mean if you are innocent what do you have to hide?

And all of the talk that this is related to terrorists: If there were recognizable phone numbers for actual or suspected terrorists, then going through the time-honored process of showing probable cause in front of a dispassionate judge is not a burden.

Have I missed something? Or in this paranoid scramble to insure the uninsurable (i.e. guaranty that there will not be another terrorist act in the US), shall we destroy those concepts that separate us from the bombers. Perhaps there has not been another terrorist attack because the terrorists achieved their ultimate goal of changing our society and form of government.
6.1.2006 3:00pm
M. Simon (mail) (www):
Here is what one libertarian thought of licensing:

"Unless we put medical freedom into the Constitution, the time will come when medicine will organize an undercover dictatorship. To restrict the art of healing to one class of men, and deny equal privilege to others, will be to constitute the Bastille of medical science. All such laws are un-American and despotic, and have no place in a Republic. The Constitution of this Republic should make special privilege for medical freedom as well as religious freedom." abridged quote --Benjamin Rush, M.D., a signer of the Declaration of Independence
6.1.2006 3:10pm
M. Simon (mail) (www):
Sorry,

I was supposed to be in the marriage thread.
6.1.2006 3:14pm
Freder Frederson (mail):
The stronger reason is that even if the current administration is pristinely innocent of any abuse, the next administration may not be. Nor the next after that.

My point is that this administration is supremely corrupt and full of liars. There is no reason to believe that they are "pristinely innocent of any abuse". In fact they have proved over and over that they will exagerrate, obfuscate, and outright lie to achieve their goals and do what they want to do and that the have absolutely no respect for the constitution, the rule of law, or international norms of human rights.
6.1.2006 3:25pm
Jeek:
Republicans may rue the day that they gave government powers that they would rather Democrats not have.

Why? How are the government's current powers inhibiting, coercing, silencing, or punishing the Democrats? All the egregious powers this supposedly ruthless and evil administration has don't seem to have been doing it any good. Is your argument that the Republicans will rue the day the Democrats have great power, because the Democrats will actually know how to use it competently and effectively?
6.1.2006 5:01pm
Grover Gardner (mail):
"My point is my privacy is breached by a number of institutions. To declare that, since my mortgage info is public record I'm not supposed to worry is nuts."

Richard, I think you completely missed my point. Any commercial transaction you make is a matter of public record. Your private phone calls are not. A commercial institution cannot tap your phones to find out if you are worried about your health or your financial state. And despite what seems to be a plethora of information about you available to commercial interests, there is still a level of protection against unwonted intrusions into your private life.

"The point is, with all this crap floating around out there on me, and with the charitable presumption that other stuff that's not supposed to be floating around--like my IRS stuff--isn't, why is the HSA's work the major threat? "

Because, as I said before, there's a certain level of privacy beyond which commercial institutions cannot penetrate without violating the law. You cannot be denied an insurance policy based on the content of a private phone call with a family member or friend. There is also a certain level of privacy beyond which the government cannot penetrate without probable cause. Of course your medical records are, to some extent, public information. But a cop can't go into the hospital and riffle through the files to see who might be beating his wife or abusing his children. Your home cannot be searched on the *off chance* that you might be committing a crime. Likewise, a law enforcement official must have a warrant to obtain your private phone records. He can't just have a "looksee" to find out if you're up to something suspicious.

So why is the NSA's search of phone records a problem? Because it has been in the past. People in our government have abused their authority and placed the safety and well-being of innocent private citizens in jeopardy. It's a matter of fact. So certain procedures were put in place to try to avoid such things in the future while still granting law enforcement the power it needs to do it's job. It's that simple. All that is asked is that those statutes and protections be observed. That's all. If people in power think they can bypass these rules, then we have a problem. It's just that simple.
6.1.2006 7:21pm
Just an Observer:
Medis: One of the oddities in the Administration's position is that they have in fact effectively claimed that there is enough evidence at the time the surveillance starts to support getting a FISA order. But they appear reluctant to allow the FISA court to review that claim after-the-fact in certain cases.

I don't think the administration actually has made that claim, although officials such as Gonzales are happy to let us form that conclusion from their obfuscatory remarks. My own surmise is that many or most of these fishing-expedition intercepts inherently could not meet FISA's requirements, regardless of how easy the prodedural paperwork is made or how much time is allowed for retroactive warrant applications.
6.1.2006 8:10pm
Fub:
Jeek wrote:

Republicans may rue the day that they gave government powers that they would rather Democrats not have.
Why? ...Is your argument that the Republicans will rue the day the Democrats have great power, because the Democrats will actually know how to use it competently and effectively?
There's an old political apothegm that partisans too often forget. It goes something like: When your party is in office never give government a power that you don't want it to have when your party is not in office.

The reason boils down to: You may trust your party (or yourself in the case of the executive) not to abuse some unreviewable power, but when you are not in office can you trust your opponents not to abuse it?

I'm suggesting that the present NSA domestic phone call data mining with no judicial or legislative oversight is one of those powers. Just for hypothetical example, what might some future Democrat version of Richard Nixon do with that unreviewable power?
6.1.2006 8:43pm
Richard Aubrey (mail):
Grover. You miss the point, again.

It matters not whether the stuff is public record. My privacy is at risk or it is not. Doesn't matter that the law says it's public record. If my phone calls are supposedly private and then the law changes to make them public, do I have to stop being annoyed?

The law in this case makes artificial distinctions. It is irrelevant to me and my privacy concerns which side of the line my concern lands on.

And, to return to my original point, considering all the stuff that's known about me on the public record, and all the stuff that's supposed to be private, except for browsing government employees, why am I supposed to be worried about data-mining?
6.1.2006 9:33pm
Grover Gardner (mail):
"It matters not whether the stuff is public record. My privacy is at risk or it is not."

You can't seem to distinguish between matters of public record and your own privacy as an individual. When you sign a contract with a financial company you are allowing them to share your financial data. That's what the contracts says. It doesn't give them the right to monitor your phone calls or come to your house demanding payment whenever they like.

Look, if you want to fly under the radar it's perfectly easy. Millions of people in this country do it every day. But if you choose to buy a car on time or take out a mortgage or have a credit card, you've agreed to have your information shared with others.

But your local police department cannot tap your phone without a warrant or search your house whenever they feel like it.

"And, to return to my original point, considering all the stuff that's known about me on the public record, and all the stuff that's supposed to be private, except for browsing government employees, why am I supposed to be worried about data-mining?"

One, because a company who checks out your credit report or tracks your purchasing preferences is not seeking to incriminate you. Two, because data mining is another matter altogether and can potentially ensnare innocent people and ruin their lives. Theree, because our government is not supposed to spy on it's own citizens. Thta's one of the fundamental tenets of our Constitution. It's there for good reason because our founders came from a country where an absolute monarch could imprison dissenters on a whim. But don't worry about it, for all I care. Worry about a President and an AG who argue that they don't have to obey the law. And try to understand that it's not just about YOU.
6.1.2006 10:35pm
Just an Observer:
Phone-call records are not "public records" by a long shot, just because their privacy interest does not rise to the level of meeting a Fourth Amendment test for "reasonable expectation of privacy."

In fact, federal law accords these records statutory privacy protection, and forbids telecom companies from providing them to any government agency except under carefully defined circumstances.
6.1.2006 11:23pm
Enoch:
I'm suggesting that the present NSA domestic phone call data mining with no judicial or legislative oversight is one of those powers. Just for hypothetical example, what might some future Democrat version of Richard Nixon do with that unreviewable power?

Probably something like what Bill Clinton did with the power of Echelon. Oh wait, I forgot, US government data mining was invented on January 20, 2001.
6.1.2006 11:26pm
Enoch:
The reason boils down to: You may trust your party (or yourself in the case of the executive) not to abuse some unreviewable power, but when you are not in office can you trust your opponents not to abuse it?

I'm suggesting that the present NSA domestic phone call data mining with no judicial or legislative oversight is one of those powers.


The "outrageous new powers" the Republicans have gained since 2001 do not seem to have created any obvious political advantages for them. Thus, either these powers are not very exciting / useful for internal political purposes, or the Republicans are too timid or too incompetent or too ethical to use these powers effectively. If the first case holds, then the Republicans have nothing to fear when the shoe is on the other foot and the Democrats have them.
6.1.2006 11:36pm
Grover Gardner (mail):
"Probably something like what Bill Clinton did with the power of Echelon. Oh wait, I forgot, US government data mining was invented on January 20, 2001."

Oh. Bill Clinton did it so it's okay. Glad we got that straightened out.

"The 'outrageous new powers' the Republicans have gained since 2001 do not seem to have created any obvious political advantages for them."

What difference does it make? If I break the law but don't gain anything by it, I've still broken the law.
6.2.2006 12:13am
Grover Gardner (mail):
Jeek--

"I question your assumption that the NSA does not have a very clear idea of who and what they are looking for, and are 'randomly sifting through data searching for apparent patterns generated by people they don't know.'"

Regardless, please see the paragraphs following those Enoch quotes in the article he linked to.
6.2.2006 12:16am
Richard Aubrey (mail):
Grover. The various folks you mention plowing through public records of my private affairs might not be trying to incriminate me. But they can ruin me.

And the "so Clinton did it so it must be okay" crap is over.

Perhaps you didn't see my explanation. Or perhaps I didn't post it on this board.

The reason for pointing to an earlier nefarious action is not to say two wrongs make a right. That's crap and you know it. But the logical extension of the argument seems to have escaped you. It is so obvious that it isn't spoken. That seems to have been an oversight. Which I will address:
If you weren't worried about it when Clinton did it, and you're worried about it now, you are not worried about it on principle. You're committing partisan hackery supported by double standards. No principle is involved. Clear, now?

This does not apply to you alone, of course. It's quite universally applicable. But in order to avoid confusion, I'll add it explicitly in the future instead of expecting it to be understood. Least I can do.
6.2.2006 12:25am
Grover Gardner (mail):
"Grover. The various folks you mention plowing through public records of my private affairs might not be trying to incriminate me. But they can ruin me."

Jeebus. Let's try again, Richard.

Our Constitution does NOT guarantee you the right to borrow money. You have to EARN that privilege. If you want something from someone, you sign a contract, and if they say they're going to sell your data to someone else for marketing or credit reporting purposes, that's what you must agree to. Lending laws provide you SOME protection from predatory practices, but if you screw up and mismanage your affairs, yes they can prevent you from borrowing money again or even opening a bank account. If you engage in a VOLUNTARY, PUBLIC transaction with a financial company, that is the risk you take. Certainly it's annoying to have your every PUBLIC transaction held against you, but that's the way financial companies operate nowadays.

Our Constitution DOES guarantee your right to personal privacy. No one can infringe upon your personal privacy without your say-so, UNLESS they have probable cause to suspect that you have committed a crime.

"If you weren't worried about it when Clinton did it, and you're worried about it now, you are not worried about it on principle."

Clinton did NOT authorize warrantless searches. Some think the Clinton administration conducted domestic spying. George Tenet and Michael Hayden testified in 2000 that Echelon was not used for domestic tapping. Some people believe that the US and the UK shared information in a way that would end-run the strictures against domestic spying. This is possible. There was a great deal of interest at the time as to whether Clinton was going too far. But the issue has gained more prominence with the NYT article and the Bush Administration's admission that there is data mining taking place and that it involves US citizens only. Together with the Bush Administration's other legal justifications for their actions, this issue has gained prominence. It doesn't exist in isolation. Many people see it as an attempt to restore a level of power to the executive branch which was abused by previous administrations.

I think Bill and Hilary did some pretty shady things while in office. It doesn't rise to the level of what I perceive to be this administration's disdain for oversight and accountability. If that's partison, so be it.
6.2.2006 12:52am
Grover Gardner (mail):
And I might add--I never said I *wasn't* concerned that Clinton did it. I have said that previous administrations have attempted to abuse their power to spy on citizens, and that's what I meant.
6.2.2006 12:58am
Christopher Cooke (mail):
I don't mind the data mining so much as the warrantless wiretapping that may follow it.
6.2.2006 3:30am
randal (mail):
I think you all are way overthinking this. Maybe they're not doing anything with the data. Who knows? But of course they want it, just to have it. That would be a great database to have for all sorts of purposes.

Total Information Awareness was a clue: Information at the Executive's Fingertips is the goal. Erode warrant requirements, condition companies to hand over data, nurture fear within the privacy vs. security propagandebate, and start building databases. That's step one.
6.2.2006 5:21am
randal (mail):
"The various folks you mention [anyone with access to public information, i.e. the public] plowing through public records of my private affairs might not be trying to incriminate me. But they can ruin me."

Then you better get an accountant and a lawyer, and get your affairs in order. Sounds like you have a personal problem that's not related to privacy policy generally.
6.2.2006 5:36am
Enoch:
What difference does it make? If I break the law but don't gain anything by it, I've still broken the law.

That was not the issue. The issue was "oh won't those foolish Republicans be sorry when the monster they've created is under the control of the Democrats". My contention is that the Republicans are not going to be sorry, since they appear to have been unable to use their ill-gotten powers for any domestic political advantage.
6.2.2006 7:26am
Richard Aubrey (mail):
Gordon. You didn't say it was okay when Clinton did it. You sneered that the clinton-did-it defense was silly. The logical conclusion is the two wrongs don't make a right defense was your point. There would be no other point.


You still aren't getting my point. I'm not asking whether something can be said to be legal, or illegal, depending on what I want my hired gun and well-shopped judge to find. Ditto constitutional.

I'm saying, given the stuff known about me already, what about the data-mining process (given what we know about it and not lying and calling it domestic warrantless wiretapping) is worse, or even as bad?

Now, you can get all breathless about the constitutional issue, which means that as soon as SCOTUS, or some lower court finds it's constitutional, you are completely without an argument.

But your being completely without an argument does not change my question of what about data-mining is worse than what is already going on?
6.2.2006 8:59am
Grover Gardner (mail):
Richard, I already answered that question. See my earlier response.
6.2.2006 12:41pm
Philip F. Lee (mail):
Let me try an abuse that we can all understand.

Popular and witty young Senator Bennigan has decided to run against the current VP for President of the US. The present VP has ethical challenges and is not particularly popular or witty. We'll call him VP Fixum.

Sen. Bennigan was sent by his personal physician to see a cartiologist because a heart murmer was detected in his last physical. Sen. Bennigan wishes to keep this information private, so he made appointments and held discussions on his personal cell phone. A number of calls to the cartiologist was necessary for coordinating Sen. Bennigan's appoints for stress tests and other heart function tests.

It turns out that the murmer was of no significant effect. A close ally of VP Fixum in NSA noticed the pattern of calls by monitoring Sen. Bennigan's cell phone and makes the connection between the cartiologist and the Senator. He informs his friend VP Fixum who decides to release a warning to the press just days before a crucial primary which could sink Sen. Bennigan's race if he loses. The warning will be in the form of a rumor from a source not tracable to VP Fixum that Sen. Bennigan has been seeing a heart specialist for a severe heart problem.

Now, how will that abuse of the election process be detected? Prevented? And don't tell me it will be by the release of the Senator's medical records because it is questionable whether there is time or ability to recover in a tight race with a charge like this in the last week of a campaign.
6.2.2006 1:34pm
Richard Aubrey (mail):
The abuse could be a problem. But that could happen with public records we generally think of as more or less private--because we would prefer it that way.

In one of his books, Robert Bork quoted a newly-appointed federal judge who had encountered a member of SCOTUS. "Glad to meet you. I've just sworn to uphold your next whim." Being a judge and all, he said "whim" instead of "brain fart"

During the Mass, the priest holds up the Host and consecrates it and it becomes, in the view of Catholics, The Body of Christ. That's not what happens to an issue when SCOTUS generally assembled has a collective brain fart. What we have is settled law until the Supremes' luncheon menu changes and then...it's not.

My point is that if all you have to complain about data mining is its possible constitutional problem, you don't have an argument next time the Supremes have a different opinion on the subject. Or an opinion, since they don't have much of a record on this new-fangled technology.

You need to have a reason to dislike it that will last longer than the possible unfortunate SCOTUS decision.

The abuse mentioned above, reference phone calls to the doctor, is one reason. Problem is, it can already be done without data mining. And, probably, legally. Thinking of smooth-talking a receptionist or using a PI.

And timing is crucial. Rather went with the TANG docs too soon. Their al Kaka ammo dump story would have been timed to be beyond repair except the NYT got anxious and went off a week early. And that's all legal, except for forging federal documents, which would probably be prosecuted if anybody could find Lucy Ramirez. All of which is to say, what's NEW about data mining?
6.2.2006 2:44pm
Philip F. Lee (mail):
Aubrey writes:

The abuse mentioned above, reference phone calls to the doctor, is one reason. Problem is, it can already be done without data mining. And, probably, legally. Thinking of smooth-talking a receptionist or using a PI.


And so it can and has -- you might remember a candidate by the name Thomas Eagleton -- although not with a heart problem. So, what? Your argument appear to be that we ignore the increase in capacity for abuse because alternatives exist.

That argument doesn't sit well with me.

My question is: What means do we have for limiting and discovering abuse by government with this new monitoring function?

Until there are some checks and balances in this program that are public and seen to work, I'll suspect the worse.
6.2.2006 7:36pm
dfolds:
Your analysis of the tradeoff is basically accurate, but it is a straw man. A program such as the one you describe is worse than useless, because it diverts too many resources from more productive investigations. There is no reason to believe such efforts are ongoing. If there is any credibility at all to what we've read about this program, all we can infer is that a massive database of communications data is being accumulated and maintained. (The phone companies already maintain such data, but do not keep it permanently.) The existence of this database would allow social network analysis techniques to be used to find candidate associates of a suspect person. The data, if maintained for a long period of time, will assist in identifying facilitators and other types of supporters both in and out of the US. Once a phone number associated with a suspected terrorist is identified, pattern-matching techniques can produce high-likelihood matches with associates who are networked with the suspect. The value of having such data on hand, available to the investigators on short notice, seems obvious. The tradeoff is not between privacy and security. The tradeoff is how much we trust, versus distrust, our own government. Certainly, it is possible for such data to be used for illicit purposes -- but the risk of misuse of telephone calling pattern data seems small compared to the benefit.
6.2.2006 7:57pm
statfan (mail):
Feverishly I check the papers for tales of unfortunates who, due to NSA, have been deprived of privacy/liberty.

What about Maher Arar?

His detention was precisely due to these sorts of six degrees type searches.
6.5.2006 4:31pm