I've noted before that many federal courts are ignoring the language of amended Federal Rule of Evidence 702, and relying on selective quotes from earlier precedents to evade their gatekeeping responsibilities. An excellent example just came across my desk, Liquid Dynamics Corp. v. Vaughan Co., Inc., 449 F.3d 1209 (Fed. Cir. 2006).
A few interesting, and disturbing, things about this opinion:
(1) The court never cites the text of Rule 702, or, for that matter, shows an awareness that Rule 702, as amended in 2000, is the governing rule for the admissibility of expert testimony;
(2) The court cites the 1993 Daubert opinion as the apparent last word on the scope of Rule 702, even though Daubert was only the first in a trilogy of relevant cases ending in 1999, and Rule 702 was amended in 2000 to codify what amounts to a strict interpretation of all three opinions, resolving certain ambiguities in the trilogy in favor of a conservative admissiblity standard;
(3) A very brief excerpt: "The appellant argued that the expert used incorrect data or was missing data to run the CFD software and used the wrong equations to run his CFD analysis of the engine's aerodynamic properties. Such a flawed analysis, it argued, made the testimony and evidence unreliable." The court concludes that this objection goes to weight, not admissibility, and refuses to review the reliability of the evidence. Yet Rule 702, as amended, specifically states that expert testimony is only admissible if "the witness has applied the principles and methods reliably to the facts of the case." If an expert used incorrect or missing data and the wrong equations in an analysis, it's hard to see how he met the standard imposed by the above language, and the court certainly doesn't explain it.
(4) The court cites Daubert for the proposition that "the focus of a court's inquiry into the relevance and reliability of scientific evidence 'must be solely on principles and methodology, not on the conclusions that they generate'", but fails to recognize that, even if one wrongly ignores the text of Rule 702, the 1997 Joiner case specified (specifically in response to the misuse of the earlier language in Daubert) that "conclusions and methodology are not entirely distinct from one another," and that courts could reject testimony even when based on what, in general, may be a reliable methodology, if it was misused in the case at hand.
(5) The court cites a 1986(!) 8th Circuit opinion for the proposition that if inadequacies in expert testimony, especially if they can be vigorously contested at trial, are a matter of weight, not admissibility. In terms of the evolution of federal expert evidence law, 1986 might as well be 1800.
(6) In fairness, the court cites a favorable 2003 11th Circuit opinion, which unfortunately is equally wrongheaded, showing how judicial error can compound judicial error. The 11th Circuit opinion bizarrely actually does quote the language of amended Rule 702, and then proceeds to completely ignore the requirement that "the witness has applied the principles and methods reliably to the facts of the case." (And the 11th Circuit opinion is also the apparent source of the 1986 citation.)
[Update: A federal district court recently wrote, "Rule 702 of the Federal Rules of Evidence, as discussed and interpreted by the Supreme Court in Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993) and Kumho Tire Co. v. Carmichael, 526 U.S. 137, 119 S.Ct. 1167, 143 L.Ed.2d 238 (1999)." It would have been pretty difficult for the USSC to have discussed and interpreted current Rule 702 in these cases, because they were also decided before current Rule 702 existed. I wonder if many federal judges are simply unaware that Rule 702 was amended in 2000. If so, it would behoove attorneys seeking to enforce the Rule to remind them.]
The relevant excerpt of the Federal Circuit opinion can be found below:
(show)
Yes, judges lounge around chambers looking for ways to "evade" their responsibilities.
I think that in this case, Vaughan's argument only goes to Step 1 (sufficiency), but is "sufficiency" synonymous with "most accurate"? I honestly don't know the answer to that, but if the applied data was sufficient to model the actual behavior of the structures in question, I'd be happy to argue that the data was "sufficient." As well, there doesn't appear to be any debate that Lueptow applied the "principles and methods reliably." He applied CFD analysis reliably, as Vaughan admits; Vaughan disputes the input, not the method or the output.
Also, at what point does scholarly interest become unhealthy obsession? Someone convinced of the obvious correctness of his position in spite of considered contrary opinions is a partisan, not an intellectual.
In my experience they frequently do. At a recent CLE hosted by the Faculty of Federal Advocates, a Magistrate Judge informed the audience that it was "a waste of time" to file motions for partial summary judgment. Many of the US District Court judges are also now adopting house rules that radically limit the ability to pursue summary judgment. E.g., a 20 page (double-spaced, natch) limit *including* the statement of undisputed facts.
Maybe you should send a copy of this to your soon-to-be-former colleague so that she can resolve this issue when it comes before a panel that she is on... or perhaps submit an article on the Federal Circuit's misuse of the Fed. R. Evid. to GMU's own Fed. Cir. B.J.?
Or how about, "A review of the caselaw after Daubert shows that the rejection of expert testimony is the exception rather than the rule. Daubert did not work a ''seachange over federal evidence law,'' and ''the trial court's role as gatekeeper is not intended to serve as a replacement for the adversary system.'' United States v. 14.38 Acres of Land Situated in Leflore County, Mississippi, 80 F.3d 1074, 1078 (5th Cir. 1996)."
Yet more, "proponents 'do not have to demonstrate to the judge by a preponderance of the evidence that the assessments of their experts are correct, they only have to demonstrate by a preponderance of evidence that their opinions are reliable. . . . The evidentiary requirement of reliability is lower than the merits standard of correctness.'"
I'll stop there. I just fail to see the "strict interpreation...in favor of a conservative admissibility standard" from the comments. Could be that I don't have the Bernstein edition of the FRE.
Pot:
I probably know just slightly less about this case than anyone else commenting on it, but a key component of your criticism (item 3) seems to be misdirected given that it was a citation of a different case.
Well that could mean that every paramater of every aspect of the principles and methods is demonstrably free of error, or it could be that the expert applied a the principles and methods to answer the kind of question for which the principles and methods can be relied upon to answer.
In other words, an expert on ballistics applying his knowledge might opine as to what kind of gun likely fired a bullet. The defendant might complain that the expert's opinion is flawed because of how he used the available data, but the testimony is of the type that ballistics science can be relied upon to support. If the same expert applied his knowledge to opine as to what color shoes the perpetrator was wearing, one could reasonably conclude that even though the expert 1) had sufficient facts to analyze the bullet fired and 2) used reliable principles to analzye the bullet, the testimony is inadmissible because 3) the expert did not apply the principles reliably - i.e. to answer a question that the principles are capable of answering.
Here, the excerpt indicates that the likened the testimony to "generally reliable scientific evidence." What more do you want?
Maybe calling this an obsession is a stretch, but you appear to be incapable of recognizing that another interpretation is possible. Simply insisting that the language is "clear" doesn't make it so. Your interpretation is plausible, but it is not the one that courts are applying. Moreover, your interpretation appears to be the product of not having practiced, and leads to absurd results: the opposing party will always be able to quibble with how an expert applied the principles and methods. If judges resolve these disputes, then there will be nothing left for the jury. Once the judge decides that one side's conclusion is reliable, the other side's conclusion in many cases will necessarily be unreliable, because the disputes about methodology and use of data that otherwise come out in cross--that the jury relies on to determine whether the expert is credible--would already be decided.
As for the interpretation "it could be that the expert applied a the principles and methods to answer the kind of question for which the principles and methods can be relied upon to answer" this is precisely the "methodologies only" logic rejected by the Court in Joiner, and even more emphatically rejected by 702.
I only read the excerpt provided, and did not see the transcript of the original trial proceedings or the pleadings in this case. From the excerpt provided, it does not seem self-evident that Vaughan demonstrated that this was a "garbage in" scenario. I am not an expert in CFD analysis or helical flows, so perhaps someone who is could let us know whether the differences in Lueptow's models make his analysis "garbage," or are merely akin to rounding pi to 3.14159265.
"But nothing in either Daubert or the Federal Rules of Evidence requires a district court to admit opinion evidence which is connected to existing data only by the ipse dixit of the expert. A court may conclude that there is simply too great an analytical gap between the data and the opinion proffered."
This is exactly consistent with my comment above.
Ballistics evidence &methodology --> type of gun opinion = not too great an analytical jump
Ballistics evidence &methodology --> color of shoes worn by shooter opinion = too great an analytical jump
He's saying that the courts aren't applying it correctly, and you guys respond by saying that this is how the courts apply it.
The textual mandates of Rule 702 are so plain as to obviate any need to consult such interpretive aids as the Advisory Committee Notes? And yet to sell the interpretation, it's necessary to disregard the text, in favor of vacuous and inflammatory rhetoric that we're to pretend is synonymous with the text, like "garbage in, garbage out"?
The whole problem may be with the draftsmanship of the Vaughan opinion, which may not have recited the underlying facts sufficiently? And yet this argument is somehow about Rule 702, rather than (say) the need to abolish unpublished federal appellate decisions, where a lack of such elaboration is the rule rather than the exception?
But that may not be your primary interest, and in any event you've answered my question. You seem to be saying that the written analysis was so poor as to leave you with the impression that even if the correct result was reached, it may have been by chance alone. How far does that view depend, I wonder, on your interpretation of both the Vaughan court and the 11th Circuit's decision as holding that "no matter how bad the underlying data, it's an issue for jury weight"? I ask because I'm pretty clear in my own mind that neither court held that. But maybe your issue has less to do with the nominal holdings in the cases than with a lackadaisical attitude toward gatekeeping that you believe the opinions to betray?
I don't mean to cross-examine. No doubt that's rude. But I keep trying to define the grievance, and I can't quite shake the feeling that as much as anything else, it's dissatisfaction with the abuse-of-discretion standard for appellate review, under which Daubert rulings are affirmed perhaps 90% of the time. Both Vaughan and Quiet Tech, for instance, were affirmances. Leaving to one side the whole debate about what the plain text of amended Rule 702 does or does not mandate (a more uncertain issue than you're making it, I think), it seems as though your underlying issue is that the appellate courts should more aggressively enforce a more vigorous scrutiny by the district courts of whether experts have applied possibly valid methodologies in unfaithful fashion. If that's the issue, then I might agree or disagree about the policy goal, while still doubting that the plain language of Rule 702 tells us very clearly how intense that scrutiny should be. Possibly we might also disagree about whether such scrutiny is currently occurring at all in satisfactorily consistent fashion, though we might still reach agreement on the point that under Rule 702, *some* such scrutiny should.
You identify many legitimate problems with Rule 702. My problem with the Federal Circuit is that it never identifies or applies the proper legal standard, which is that the district court has to ensure that "the witness has applied the principles and methods reliably to the facts of the case." It's possible that the evidence was sufficient on that score to allow admissibility. It's also possible that applying the abuse of discretion standard, the court would have been right to defer to the district court. But I read the opinion as endorsing the view that so long "as the underlying methodology is reliable, the testimony is reliable, regardless of how it was applied in this case," which I think is precluded by Rule 702, and strongly discouraged even by Joiner. It's possible that the underlying problem is that appellate courts figure that the abuse of discretion standard is so liberal why bother going into a deep analysis; it's also possible that the parties bear some responsibility for not briefing the issue well (Westlaw has the briefs, but I haven't looked at them). Regardless, isn't it troubling to see a case with a reasonably lengthy discussion of the admissiblity of expert testimony that cites pre-2000, and even pre-1993 precedents, but never refers to the language of the governing rule?
The court concluded: "We conclude that a reasonable juror could consider Lueptow’s testimony explaining the very robust, helical flow in the models and infer that the similar accused tanks will produce flow similar to the modeled flow."
In other words, the court concluded that the expert's testimony, and the methodology it was based on was reliable enough that the jury could consider it.
I think your real problem is that the court didn't quote your favorite words from rule 702, even though it did analyze whether the testimony was sufficiently reliable to be shown to the jury.
Also, I would like to know how my previous analysis is inconsistent with Joiner.
And one last thing, which I'm surprised you don't know: Insisting many times that something is clear is less persuasive than explaining it once. You also gain credibility when you admit to the strengths of opposing arguments when they are legitimate.
I think that I'm less disturbed than you because at bottom, we don't agree that the opinion neither identifies nor applies the proper legal standard. Maybe we're just speaking different languages here. You think (I'm gathering) that the opinion's failure to provide a satisfactorily complete and integrated argument justifies (or is almost definitionally equivalent to) that charge. I see ways (perhaps many of the same ways as you, if we went through the text word for word) in which the opinion fails of perfect completeness and integration, but to me that's just a point of departure, leading me to read it again with the goal of giving it the benefit of the hermeneutic doubt. I would charge the court with ignoring the applicable legal standard only if I thought, after long and serious study, that there were no other plausible reading. (No doubt that's partly because I'm in practice and need to manifest a basically positive attitude toward the judiciary.)
Anyway, when I reread the opinion with that mindset in place, I find myself distinguishing between talking the talk and walking the walk. On talking the talk, you're right that the opinion doesn't go much beyond citing Rule 702, without quoting either it or Joiner. That would be more disturbing to me if the opinion went on to utter dismissiveness of the issues about the data. But on my reading, it doesn't. It may give them short shrift, in your view. But after all, the only reason you and I know about them at all is that the panel wrote about them. The opinion does pay *some* analytical attention to whether the data issues should be fatal -- albeit not enough attention, perhaps, to permit the opinion reader to come to his own informed and independent assessment. That, perhaps, is part of your frustration, and I agree that it would be good if appellate opinions in general (not just the ones about Rule 702) reached that aspirational level of transparency. I just don't see the Vaughan opinion as an unusually egregious offender. And given the standard of review for Daubert rulings, which arguably gives appellate courts some license to stop short of discussing *every* little thing, I don't think you're likely to get a lot happier with the appellate opinions anytime soon, on that front. Especially not, perhaps, in patent cases whose technical details are (to me, anyway) somewhat inscrutable.
Meanwhile, for a less than fully explanatory appellate opinion where the standard-of-review shoe was on the other foot, and where I'd suspect you'd probably say that things worked out in a basically good and proper way (as would I, on the whole, even though I couldn't prove it beyond all peradventure from the text of the appellate decision), see Kilian v. Equity Residential Props. Trust, No. 04-16723 (9th Cir. June 30, 2006) (unpublished).
Cheers,
Peter
Anyway, I don't fault Rule 702 for failing to draw that line in some neat little algorithm, just as I don't fault the Vaughan opinion failing to do so, because the line is not very easy to formulate in abstract terms of general applicability. In the end, some element of judgment is probably necessary. (As Wittenstein says, explanations come to an end somewhere.) Factoring into such judgments may be the notion that juries *can* evaluate many of the factual assumptions that commonly go into expert analysis -- as well, at least, as they can evaluate any other question of naked fact. That idea may have animated (and in its surviving form, may still find some roots in) the previous legal custom of asking experts hypothetical questions, the notion having been that if the jury wasn't satisfied that the evidentiary record established the factual premises of the hypothetical questions, it was free to reject the conclusions that the expert drew from those premises. But whatever the merits and demerits of the hypothetical question tradition, it does seem legitimate to claim that the closer something comes to a brute observational fact, the more equipped may be the jury to evaluate it.
That does leave the mixed case, confessedly, in which the jury is ill-equipped to evaluate the ways in which some degree X of inaccuracy in the expert's assumptions will *affect* the conclusions. How sensitive, for example, are the results generated by CFD equations to different parameter values? There is a whole branch of learning, I believe, that purports to be able to measure such things, but sometimes, even if he is not himself a specialist in the arcane discipline of sensitivity analysis, the expert himself may be able to offer a reliably informative judgment. (If not, he arguably shouldn't be playing with those computer programs in the first place.) And that still seems to me to be what happened in Vaughan. One thing I do know is that few scientists would claim that an analysis automatically becomes seriously suspect whenever any inaccurate, undocumented, or controversial value happens to be given for a parameter. Newton didn't get all the equations about gravitation exactly right, but he came close enough for most government work.
I'm also starting to suspect that if you wanted to make your case about the waywardness of the federal courts, you might do better to focus on district court decisions, where the standard of appellate review isn't a complicating factor.