I've posted on SSRN (and sent to law reviews) my paper, "Expert Witnesses, Adversarial Bias, and the (Partial) Failure of the Daubert Revolution." It's already received some attention from Blog 702 and Law.com Here's the abstract:
This manuscript raises two questions that have been surprisingly missing from the voluminous law review literature on expert evidence since the landmark Daubert decision. First, what is the underlying rationale for the replacement of the old qualifications-only, let-it-all standard for expert testimony with Daubert/Federal Rule of Evidence 702's requirement that all expert testimony be subject to a stringent reliability test? Second, once we have identified this rationale, has the "Daubert revolution" succeeded on its own terms?
I conclude that the implicit rationale for the reliability test is to preserve the perceived advantages of the adversarial system, while mitigating the harms to the courts' truth-seeking function by the inevitable strong biases that accompany adversarial expert testimony. These biases include the conscious biases of hired guns, the unconscious biases of other paid experts, and the selection biases that result from the fact that attorneys "shop" for their experts from a large pool of qualified individuals.
Rule 702 thus attempts to serve a worthy goal, but it far from fully succeeds in efficiently achieving this goal. First, in the context of forensic expertise in criminal cases, Rule 702 does nothing to address the huge gaps in resources between the prosecution and most defendants that severely inhibit defendants' ability to challenge unreliable prosecution expert testimony.
Second, Rule 702, applied correctly, does succeed in barring "junk science" causation evidence in toxic torts cases. However, it does so at the expense of excluding speculative evidence supporting causation, even when most experts in the field would conclude that the relevant evidence is a sufficient basis from which to find causation by a preponderance of the evidence. While Rule 702 is easily preferable to the prior overly permissive regime, it likely goes too far in insisting on a reliability test that makes the courtroom stricter about causation evidence than is the scientific community itself. The way around this problem is to amend Rule 702 to allow courts to admit educated guesses about causation, but only when nonpartisan experts, not subject to adversarial bias, are willing to make such guesses.
Finally, Rule 702 puts severe restrictions on the testimony of experience-based testimony by connoisseurs. Such experts may only testify if their field of expertise is a legitimate one, and they have proven to the court that they truly have the expertise they claim. Rule 702 also properly prevents attorneys from shopping for outlier and hired gun connoisseurs, given that there is no objective way for a jury to determine whether an experience-based expert's views are correct or representative of other experts in the field. Therefore, in the context of connoisseur testimony, courts should either replace adversarial experts with a panel of nonpartisan experts, or only allow an adversarial expert to testify if his conclusions are consistent with those of a nonpartisan advisory panel.
I am confused. If evidence is rightfully characterizable as "speculative," then by definition isn't it insufficient as a basis for a verdict? Doesn't there have to be more than "speculative evidence" for experts in a field to "conclude that the relevant evidence is a sufficient basis from which to find causation by a preponderance of the evidence"? Can they just "intuit" that there is causation, that is "know" it without being able to articulate the compelling logic that dictates such a conclusion? By "preponderance of the evidence," we don't mean any time statistical testing yields a p value of <.50, consistent with a result less likely than not to reflect mere chance, do we?
Could we have some illustrative examples of "courtrooms (being) stricter about causation evidence than is the scientific community itself"? When alluding to "the scientific community itself," is the implication that we are referring to the "consensus" view of the scientific community as reflected by what is in the published literature, or only that an expert can be found who will say he/she believes there is causation? What constitutes an "educated guess" as opposed to an "uneducated" one, or "speculation," or "hunch," etc.
I think the situation imagined is one like the following.
Suppose there are two widely accepted views in the scientific community about whether chemical C causes effect E. Let's stipulate that one explanation that says C does cause E is given by the jury a probability of 49% to be correct while the other widely accepted explanation that says C doesn't cause E is also given probability 49%. If this is all the evidence the jury hears they don't have sufficient reason to find that there is a preponderance of evidence that C causes E.
However, if the jury also gets to hear that a small minority of scientists believe in a third explanation that also supports the idea that C causes E then even if this explanation is marginal and they only give it a 1.5% probability of being correct it would be enough to push them over the threshold.
In other words the worry is that the jury cannot correctly take into account marginal or speculative theories that together with other explanations might justify some results.
I think this worry is unfounded for two reasons. First of all the evidence about lay people's ability to reason about probability suggests that it is pitifully bad. Consider the Kahneman and Tversky results (tho maybe they only cited) about people giving a higher probability to someone being a feminist bank teller than just being a bank teller. Worse we all know that people are very bad at dealing with small probabilities, they tend to either round them up into the plausible range or forget about them as insignificant. While this is good in daily life (you don't calculate mugging probabilities every time you leave your door) it means that even if theoretically speculative evidence might be useful in the jury/judge's decision it is more likely to cause confusion than help.
Secondly, these marginal or speculative ideas can be bunched together by the expert themselves to reach a non-marginal result. The court's allow experts to testify about the effects of chemicals on the body even if the underlying physics of how they bind to receptors might not be well known. Similarly I see no reason why the expert himself couldn't say that several theories point towards the fact that C causes E and that together they make a compelling case. The jury should never be in the position of evaluating these probabilities directly, only the credibility of the experts making the claims.
Frankly, I think we should have gone much further and required that all experts come with statistical evidence about the prevalence of the views they testify about in their field or something like this but it's a hard problem and I understand why such a big leap wasn't taken.
In particular I would move forensic laboratories and similar default experts employed in the criminal process to a neutral third party, perhaps move them inside the judicial branch itself. Of course I would still let both sides call expert witnesses if they wished but the government would be required to equalize the amount of money the public defense office and the prosecution office in a particular area receive for experts over the course of the year. Since the default forensic evidence is moved to a neutral third party this rule the prosecution would not be unduly disadvantaged by such a rule.
For civil cases I would like to see experts picked by some part of the judicial branch as well as experts from the defense and prosecution.
--
My earlier comment doesn't quite match the example used in the paper about the worry of excluding too much evidence but I think it is in the same spirit. The example in the paper is where a very rare condition results from taking a rare overdose of some drug. Presumably this in itself already gives the jury some reasonable suspicion that the drug caused the condition and the addition of an experts speculation about possible mechanisms this effect could have occurred would be enough to push them over the edge.
I am still skeptical that this result is really what is required by the applied the methods reliably to the evidence in this case rule. As long as the result is portrayed as sufficiently speculative it would seem that they have so applied the methods.
The biases may not even be right- or left- oriented. One of the judges in my community, in family court, doesn't really believe in spousal abuse. She just doesn't believe it really happens, and that most claims of it are made up or exaggerated. She also has trouble believing that it can happen in "nice" families. If she picks an expert witness on, say, the psychological make-up of the parties to a divorce, her bias might very well lead to her selecting a like-minded expert who tends to reinforce her opinion.
To paraphrase Winston Churchill, the adversarial system is the worst form of justice in the world... except for all the others.
But if 49% probability of causation on the basis of all the evidence presented by the parties does not mean the party with the burden of persuasion has failed, then the jury should be allowed to hear educated guesses about causation by those the court deems to be both "nonpartisan" and "expert"? So, the court reaches out to a "nonpartisan expert" of its own choosing and that expert will likely decide the matter with their testimony. The expert testifies that the possibility of causation has not been conclusively ruled out, maybe because a sufficiently powered study has not been done, and they think given all the relevant evidence the likelihood of causation must be very remote, perhaps 1.5%. (Or does the "non-partisan expert" testify that if evidence that would otherwise be inadmissable is taken into account, then the probability would be >50%?) And under what circumstances would scientific evidence that could be relied upon be held to be inadmissable?) Then the "non-partisan expert's" estimate of a 1.5% probability that there is causation somehow pushs the estimate of a 49% to a >50% probability, meaning causation has been shown to be more likely than not?!
I'm not sure whether my problem here is with Rule 702, the intersection of science and the law, probability theory, standards of proof, or what. But I don't understand why the parties should bring their experts on causation if the court's "non-partisan" expert will effectively decide the matter?
(Using the link, I only found an abstract. I will look again for the article itself.)
Just a poor dumb non lawyers thoughts!
The paper looks very interesting.
To therut, I wonder if your sentiments are the same for say, DEA officers who are called as experts in drug terminology. Better yet, how about an expert hired by Larry Birkhead to interpret DNA tests? What about experts who simply testify to an academic topic, such as the (un)reliability of eyewitnesses. They may be called and paid by one side.
I'm not a big fan of the "general acceptance" standard because it seems to me that it offloads too much of the courts' truth finding function. We've spent a lot of time and effort coming up with rules of evidence that are supposed to allow the jury to evaluate the issues in a neutral fashion and reach the correct result--but once the topic is at all technical, we abandon all of that and go to the peer review process?
Everything about peer review is antithetical to the rest of the FRE (adversarial setting, open process, a neutral decisionmaker, etc.). Why give so much weight to the results of such a flawed process?
But let's try an example:
The murder case turns on what a particular weapon did. I, a juror, spent three years in the field with that weapon, and the next enlistment as an armorer, fixing, modifying, spare-parting that weapon. I know that what one side says about the weapon to prove its case is not true.
Am I allowed to use that in my private deliberations?
Am I allowed to explain what I know to the other jurors?
Do I have to tell the judge and expect an alternate--whose qualification for taking my place would be ignorance--to be seated?
I can imagine the chaos that would occur in case involving a dispute about the application of climate science to contested government regulatory rules, to mass tort actions against utility or energy companies to abate pollution and recover damages, or to compel government entities to adopt new regulatory schemes to limit anthropogenic emissions. When does scientific inquiry evolve into a belief system for purposes of determining if there is adversarial bias?
Relative Risk = [Incidence rate of disease in exposed group]/[Incidence rate of disease in nonexposed group]. DB, doesn't a relative risk above two (with a suitable small confidence interval) argue for "more probably a result of exposure" than more probably not a result of exposure"?
Had the side contending that which you as someone with special expertise believe to be the truth about an evidentiary question on which the case turns, perhaps they might have used you as their expert. But jurors are not supposed to have an already formed opinion about what the outcome should be, and you think you know the "answer" before any evidence has been introduced and argued.
So if in the course of voir dire it came out that you had such expertise (perhaps you come to court in uniform and when asked your military specialty you explain that it relates to weapons like the one in this case), you would almost certainly be challenged and struck, whether for "cause" (you likely already had a formed opinion) or one side wanted you off and used one of their peremptory challenges to dismiss you from the jury pool. From the court's perspective, a potential juror without your special knowledge may be wholly unqualified to serve as an expert in the instant case, but not less qualified than you to serve as a juror.
Do you "have to tell the judge and expect an alternate--whose qualification for taking my place would be ignorance--to be seated?" You have to answer honestly the questions put to you, including general ones like whether there is anything that would cause you to have an already made up mind about what the jury is expected to decide.
The case had to do with misbehavior (non-medical type) of a physician. Two nurses were allowed to remain, although one allowed that if you let docs get to you, you couldn't do your job.
However, nobody's perfect, not even trial lawyers or prosecutors. I was thinking of what would happen if they did not think to ask the most unlikely question, which is, is anybody here an expert on such&such a firearm. In other words, let's presume I get through the challenge process and, much to my surprise, find the case turns on what a particular firearm does or doesn't do.
Or, let's say my previous job was in the R&D of a particular industrial chemical manufacturer, as a tech, not a Permanent Head Damage type. And the attorneys fail to put two and two together, or maybe are not aware of one of the twos.
Let's just say I got on the jury perfectly legitimately but find I am now an expert. Now what? We have dueling experts on the stand. I know one of them is right and the other couldn't be more wrong. Or the erroneous view is conceded by the other side through ignorance and I'm the only one in the entire courthouse who knows better.
It would seem hard on the idea of justice for me to let an error govern the outcome.
On the other hand, one of the attorneys would be dumping me for the express purpose of getting a falsehood over on the jury. That doesn't strike me as particularly slick, either.
Remember, no matter how sure you are you're right, there's almost certainly somebody else out there equally sure you're wrong.
Now, if it really is an actual fact which you actually know as opposed to believe strongly, then sure, I can understand that you would find it unjust to vote to convict based on the erroneously presented evidence. You have several other options. You can disclose to the judge that you were not aware at the beginning of the trial that the case would hinge on your special expert knowledge. Now that it's clear it does, you have found that you are not capable of following the judge's instructions because of this knowledge. While you're at it, feel free to blame the idiot/lazy lawyers who wasted everybody's time by not bringing this up in voir dire so you could have saved everybody the time and effort.
Really, that's your best bet. You could also simply keep quiet, vote your conscience, and let that result, presumably, in a hung jury. That would also be morally acceptable.
But I don't think it would be morally acceptable to appoint yourself in the jury room as the supreme expert. As noted before, you may not actually be right. Even if you're right about the general point, there may be something about the case or the gun of which you are not aware. Suppose the issue was how much force it takes to pull the trigger on a particular gun. You may know from your own experience that that model only takes 5 pounds of force, while the prosecution expert says it takes 15 pounds of force. The issue is whether the gun could have accidentally gone off or not, and guilt or innocence (or murder or manslaughter) hangs in the balance; at 5 pounds, the petite woman on trial could have accidentally pulled the trigger, at 15 pounds it must have been intentional. You "know" that that model gun really takes 15 pounds. But how thoroughly do you really know this? Did you do an exhaustive literature search? Did you personally inspect every single gun coming off the line? Do you know whether it's easy to modify that gun or not? Did you review the prosecution expert witness' research to see whether he learned of a previously unknown defect in some examples of that model allowing for less force to pull the trigger? That's just off the top of my head; I'm sure with more thought I could come up with plenty more examples.
That's why the most appropriate thing for you to do is to notify the judge when you are aware that you can't follow his instructions, because your previous specific knowledge will influence your conduct as a juror. If you think there's an injustice about to happen, you can then go talk to the defense attorney and explain to him what he's missed, or even volunteer to be his expert in the retrial.
No, the idea was the jury decides that mechanism A (say that a particular compound is absorbed through the stomach and causes DNA mutations increasing cancer risk) has a 49% chance of being correct. They also find that the theory that this particular compound isn't even absorbed through the stomach has a 49% chance of being true. This tells them that their is a 51% chance the compound is absorbed through the stomach but not that it causes cancer.
If a third theory says that the compound is absorbed through the stomach and then encourages a virus to replicate which in turn causes cancer it could push their probability that the compound causes cancer up above 51%.
WRT your example: I'm not an armorer and I haven't fired anything but a Daisy BB gun in nearly forty years. So I was using guns as a broad example. But, to take unfair advantage of your top-of-the-head example about trigger pull. They vary all over the lot, practically as they come off the assembly line.
I was thinking of something far more technical.
But it could be the case in any endeavor, any avocation or vocation.
You never know. For example, I was going around with some liberal journalists on Pressthink about a report where the NYT had a wounded soldier get the Purple Star. They didn't think there was any reason to expect a reporter or the legions of hard-eyed editors and polymath fact-checkers to actually know the correct award is the Purple Heart. It's such a niche thing, to know about Purple Hearts, at least in the NYT's newsroom and, apparently, among their readers.
So if there really are people able to function in society in jobs more challenging than journalism and who don't know what most would think is normal about one thing or another, we may have an entire jury being experts on something one of the attorneys missed bigtime on account of being as knowledgeable as NYT journalists. Or, more likely, missing something slightly more arcane and has no idea to ask about it.
That was me, not Neurodoc.
My general point was that, while the scenario, a juror with absolute actual indisputable knowledge about a contested scientific/technical point crucial to the resolution of the trial, you describe is conceivable, I think it is unlikely in the extreme to arise in real life. As a practical matter, the juror's expert "knowledge" would almost certainly not be that indisputable, and more in the realm of opinion, however well supported.
--PatHMV
In that case, it's all opinion. That I don't make my living from my knowledge may be a matter of circumstance, not of deficient knowledge.
For example, I was talking to a pathologist one fine day and he said, discovering I'd been a grunt and had taken some serious first aid courses, that it would be better for an accident victim if I came along than if he, a Doctor of Medicine. Pathologists, he said, probably exaggerating, know squat about medicine.
Which, of the two of us, is going to be dismissed from a jury in which some medical issue is likely to arise? And there am I, recalling a diagram explaining the amount of time it takes to bleed out if one or another major artery is cut (see Fairbairn-Sykes, of which I own one).
Me, I'm just a broken-down old soldier whose uniforms are considered "classic" by the surplus stores and might profitably be sold to theatrical costume shops. He's a doctor. Nobody would expect me to know anything.
Anyway, the point of expert testimony is convincing the jurors, and it's certainly possible that jurors have either decided to play the game and forget what they know, pretending that the only info that counts came from the hired morons on the stand, or dismissed it due to their own experience.
The issue isn't so much that the world is full of people who know more than hired guns who make their living telling people what they're paid to tell people. It's that the case could slide into somebody's knowledge sort of sideways.
I have a friend whose twenty-two-year old son has been horribly challenged since birth. She has a sneaking suspicion it was a matter of her taking some meds during pregnancy. She has researched the hell out of the situation, knows other parents in the same boat, and spends lots of time with specialist physicians. She's also a nurse. Suppose jurors were asked if they have any medical background. She says yes and is dismissed. Or, if she were not a nurse, she'd probably say no, and be seated. And then, hm, the case starts wandering along unlooked-for paths. And gets to her situation.
Well, enough. Thanks for your input.
I don't think your hypothetic really addresses my question about the "combinatorial" probabilities when taking bits of one and bits of another. And it doesn't recognize, or at least acknowledge, that asking NPE's to "speculate" or share their "educated guesses" with the court so it may decide admissibility on the basis of such is in effect to gut 702.
What exactly is "speculative evidence" and how does it differ from mere speculation? Suppose in a controlled clinical trial a greater proportion of subjects treated with compound A improve than do those treated with B but the difference in outcomes is not great enough to yield a statistically significant result. Would an expert be allowed to say that while it is true the scientific community wants to know that results reflect a real effect not mere chance and thus generally uses as a cutoff a p value (or other measure of statistical significance) of no greater than .05, and preferrably a much smaller value, a p of .10 does mean that there is only a 10% likelihood that the observed results came about through just chance? Then go on to argue that because a P only need prove causation more likely than not, that is >50% probable (where is that feather or nutmeg when we need it?), P ought to be able to point to the p of .10 result in support of their case? This is utterly specious reasoning, but might sound good to the unknowledgeable. And that is exactly what Daubert aims to prevent to the extent possible with its criteria for admissibility of expert testimony, as I understand it.
(I have seen Ps argue "alternative" causation theories in toxic tort cases. It strikes me as at best a very inadvisable approach since it amounts to telling the trier of fact, "See which of these you think most plausible, and we will try to persuade you that it is 50%+ probable." Not more than one of competing alternatives, if any, can be more than 50%+ probable, so arguments for one must at the same time be arguments against the other(s).)
I skipped DB's discussion of "forensic" matters and those of "connosseurship," because "toxic torts" are of most interest to me. While I appreciated the general look at 702 and sources of expert bias, the use of NPEs did not seem to me very well thought through. But perhaps I should spend more time with it.
The facts were that the doc sexually harassed women employees and tried to swap prescription drugs for sex.
The defense attorney tried, during selection, to intro the thought process that people from different cultures have different outlooks--and by sneeeeeky implication ought to get a bit of a pass--but was stopped by the judge.
The facts were not in any serious dispute. Their importance was, as was the possibility of entrapment.
So that was a non-med misbehavior.
There have been a couple in our area jailed for selling scrips. One had set up in an old shoe store in our version of the inner city and had leftover funeral home chairs--a lot of them--in the waiting room. The cop who went undercover their told me the doc didn't have so much as a bandaid in the office, but did have a stethoscope, which is probably the most effective prop.