In my Iowa Law Review article, discussed in my previous post, I point out that Daubert/amended Rule 702 haven't done much as yet to improve the state of forensic science, and are unlikely to do so in the future.
One reason that forensic science (itself a misnomer, as there is little science involved, for the most part) has emerged mostly unscathed from the upheaval in expert evidence law, I suspect, is that the "Daubert Trilogy" of expert evidence cases--Daubert v. Merrell Dow Pharmaceuticals, General Electric Co. v. Joiner, and Kumho Tire v. Carmichael--on which Rule 702 is an elaboration, were all civil cases. Moreover, despite the obvious relevance to criminal cases, the criminal law bar was virtually silent when these cases were being decided. For example, when Daubert was before the Supreme Court, out of over thirty amicus briefs, not a single one was filed by an organization concerned with criminal justice issues. Almost all the energy opposing "junk science" has come from corporate America and its attorneys, with the opposition coming from the plaintiffs' bar.
Political liberals, especially elected officials (who, not coincidentally, get much funding from the plaintiffs' bar, and very little from public defenders and their clients) have generally opposed attempts to crack down on bad science (and other forms of expertise) because of the latter dynamic, and have mostly ignored the ongoing abuses of expert testimony in the criminal justice system. Prosecutors, for their part, have fought tooth and nail against attempts to restrain their use of dubious evidence, especially at the state level--I've heard from sources in several states that the main barrier to adopting a rule requiring reliable expert testimony in their states has been the strong opposition of prosecutors who fear that such a rule will make it hard for them to get convictions. Of course, their job, properly, is not to "get convictions," but to convict the guilty; and even though Daubert has actually had very little effect on prosecutors' ability to get convictions, the mere idea that their experts may be held to some reasonable standard of reliability is enough to raise prosecutors' hackles.
For reasons discussed in my Iowa article, mere tinkering with the rules of evidence is unlikely to change the current dynamic in which quackspertise is almost routinely admitted in criminal cases. But I wonder if more progress would have been made by now if quackspertise in the forensic contest had received a fraction of the attention of junk science in civil litigation, from the Supreme Court and otherwise.
For essential reading on the subject, see Paul C. Giannelli, Wrongful Convictions and Forensic Science: The Need to Regulate Crime Labs, 86 N.C. L. Rev. 163 (2007) (detailing many of the problems with forensic science) and Roger Koppl, How to Improve Forensic Science, 20 Eur. J.L. & Econ. 255 (2005) (providing the best ideas I've seen to improve forensic science, using economic reasoning). Todd recently linked to a popular version of Koppl's ideas, published in Forbes.
Related Posts (on one page):
- Some More Thoughts on Daubert and Forensic Science:
- Article on Daubert and Adversarial Bias:
I would suggest a rather revealing study: Gatowski, et al. "Asking the Gatekeepers: A National Survey of Judges on Judging Exper Evidence in a Post-Daubert World" Law and Human Behavior, October 2001.
In it, they surveyed approx 400 Daubert jurisdiction judges on whether or not they felt they were the appropriate gatekeeper, and then on the criteria they used. As one might expect, they overwhelmingly believed that they were the people who could tell good from bad science and that it was their job to do so.
Unfortunately, when asked about the specifics, they fell quite short. As the authors note:
Rhenquist and Stevens, in their dissent from Daubert warned that this would turn judges into amateur scientists, and that's exactly what has happened.
I've been in a bunch of Daubert hearings as an expert, and it has been amusing to watch the lawyers and judges stumble through technology and concepts they clearly did not comprehend, and then tell *me* whether or not what I did was "scientific."
The idea of an expert panel is not much better. It ignores the fact that there can be *real* controversy in science and that panels are profoundly subject to groupthink. I know this -- I am on three scientific standards committees and one NIJ scientific policy panel. Not only is such a constant danger, but in reading the comments to some of the standards and guidelines we have written it is clear that many of the "scientific" objections to certain things have little to do with science and have everything to do with ideology, turf, and agendas. Gatekeeping by master panels is notorious for this.
Finally, much of the discussion involving "science" in forensics has little to do with what is actually done. In most cases it takes a specific formalism, such as statistical probabilities as applied to DNA identification, and attempts to make everything else fit that paradigm. It doesn't. It doesn't in large part because most expert testimony *isn't science.*
Medical diagnosis is not science, though it is certainly an area of expertise. It is not deductive, but abductive inference. The very concept of "reasonable medical certainty" is an absurd concept forced upon us by the courts -- a kibuki theater in which the judges force us to pretend to scientific positions that we structurally do not have in order to fit a paradigm that they require.
So now, after Kumho, an expert plumber must show that somehow his or her 20 years of experience in the field represents "science" or else his or her technical expertise is inadmissable when talking about fitting a joint on a pipe.
So what happens? Everybody fakes it. There's a Daubert hearing where the criteria are not really applied because everybody knows it's BS, but everybody pretends that it isn't.
The reason that, as Gatowski et al. note, judges substitute the "rhetoric" of Daubert for the "substance" of Daubert is that if the substance were really applied, it would be absolute chaos, not better science.
In abductive inference, which is what most of this is, you do not set up an experimental protocol and a set of formal axioms and do a double-blinded study. Doin that is very important in scientific investigation, which sets up the likelihoods for individual diagnosis, but it is not done in the diagnosis itself for the most part.
Instead, you are given a set of circumstances and a set of mechanisms, and you choose among them the one that seems the most likely. That's abduction, and it invariably uses post hoc ergo propter hoc in some form. It is *unavoidable.*
To give you an extreme example, a person is observed walking on a roof, walks to the edge and falls fifty stories and splatters on the pavement. I, as a forensic pathologist, perform an autopsy and note severe blunt trauma as wellas severe coronary artery disease. Someone comes to me and says "How do you know he didn't fall because he had a cardiac arrhythmia due to his coronary artery disease, and thus died of a heart attack rather than the fall?"
The answer is that there is no way to say that didn't happen with 100% certainty -- even the presence of some bleeding upon hitting the ground does not rule it out, and sudden death due to arrhythmia often does not leave histologic diagnostic change. However, of the two scenarios, death to a fall fits the findings better.
What is the "science" that allows me to tell this? None, really. It is abductive inference -- given two scenarios, one fits the data better. Had the man been seen clutching his chest and complaining of chest and arm pain, it would make death due to a fall less likely. Had the man left a note saying his life was without meaning, it would make death due to a fall even more likely.
But the bottom line is that the kind of "Science" that Daubert reaches for is simply not part of the equation.
This kind of abstract application of inappropriate criteria for inference is mentioned quite nicely in an article some years ago in the British Medical Journal:
Smith G, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled
trials. BMJ 2003;327:1459-61.
In it, the authors note that there has never been a formal scientific double-blind randomized study that demonstrates the utility of parachutes for passengers when jumping from airplanes. Further, there are documented cases of people falling from airplanes without parachutes and surviving. Thus, by strict application of formal scientific method, there are no studies of high empiric value that demonstrate the validity of recommending parachutes for people jumping from airplanes.
They suggest that those who recommend strict application of such guidelines volunteer for the control group.
It's always stunned me that the courts allow forensic 'evidence' from tests (fingerprints, DNA, ballistics) that aren't even single-blind, when a double-blind test would be required for any kind of reliability at all. (Put that on the list of 'opinions I can volunteer to get me out of Jury Duty', I suppose.) It would seem to me that a legislative remedy to that situation would be more likely than a judicial one, though, even if it's just in the sense that 'slim' is a larger chance than 'none'...
I'm confused, too. Depending on the situation, any of these groups could be either benefited or hurt by "junk science." I don't see why any of these groups would be motivated to support or oppose the admission of "junk science" as evidence in court.
Also, David Bernstein complains about Rule 702 but makes no suggestions about how to improve it. In fact, he contradicts himself by saying,
How could "more progress" be made if "tinkering with the rules of evidence is unlikely to change the current dynamic"?
Also, in addition to Rule 702 and Daubert, another factor regarding scientific evidence should be considered -- justiciability. IMO some scientific controversies should be considered non-justiciable, e.g., the controversies over evolution and global warming. The syllabus of Vieth v. Jubelirer, 541 U.S. 267 (2004) says, "Among the tests for determining the existence of a 'nonjusticiable' or 'political' question is a lack of judicially discoverable and manageable standards for resolving the question."
No, but it would cut down on the number of people claiming that any expert who says that parachutes are useful is a lying quack because there isn't one.
In Florida State Courts, they still apply Frye and that's just NUTS!! Junk abounds, for example, in Americans With Disabilities Act accommodations determinations, it is not unusual to have the determination made by a human resource manager or marshal (law enforcement) who is the chief budget cutter, on the "common sense" notion someone "looks"
or "doesn't look" disabled. Double Yikes!!
Really, I have engaged in my own little experiment: despite the fact there is no correctable lense for me to read print (due to my autism), although my vision is cirrectable for distance for driving, a different result ensues depending on whether I wear my glasses or not. I have special sports glasses to accommodate my hearing problems from head injury (metal horse-shoe kick to head by pony) due to not being able to apply pressure of regular gasses frame over left ear. They work great, but people freak out when they see them. If I wear the sports glasses, I receive accommodations; if not, I don't -- there being NO DIFFERENCE in the fact I cannot see to read print on paper absent special features in electronic format on computer.
It makes no scientific sense, and in Florida's Courts it is really more like the roll of dice or a crap shoot when it comes to expert medical evidence. I really think Florida State Courts really need to get rid of Frye, and should be following the federal rule of Daubert/etc. in Americans With Disabilities Act cases brought under the federal statutes in State Court, but who am I to change the world?
Having read over 10,000 ADA/RA cases and law review articles/journals, I find it INCREDIBLE Title II of the Americans With Disabilities Act is not regularly being invoked in the criminal law area. What a fruitful ground to dismiss a charging document! Or to argue for/against an insanity defense.
"cirrectable " = correctable
"gasses" = glasses