Could gatekeeping for expert testimony get any messier?
Energy has been building to amend Federal Evidence Rule 702 to tighten the reins on expert testimony. The proposed amendment has received preliminary judicial approval. Congress and the Supreme Court will consider it in 2023. If each approves, which seems likely, the rule will take effect December 1, 2023.
The amendment aims to make it harder to introduce expert testimony. It will put to bed the popular observation in Daubert v. Merrell Dow (1993) about the “liberal thrust of the Federal Rules and their general approach of relaxing the traditional barriers to opinion testimony.” Under the amended rule, there would be no liberal thrust, nothing would be relaxed, and the barriers would be well formed.
Since 2000, Rule 702 has said that a witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if:
(a) the expert’s scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue;
(b) the testimony is based on sufficient facts or data;
(c) the testimony is the product of reliable principles and methods; and
(d) the expert has reliably applied the principles and methods to the facts of the case.
The proposed amendment would modify part (d) slightly to read: “the expert’s opinion reflects a reliable application of the principles and methods to the facts of the case.” The amendment’s sponsor—the Advisory Committee on Evidence Rules—says that this would direct the trial judge to pass judgment on the expert’s conclusion, not just on the opinion’s basis and methodology.
But the amendment would add another requirement. The testimony’s proponent would either have to demonstrate satisfaction of the four criteria “by a preponderance of the evidence” or, in an alternative formulation, have to demonstrate that those four criteria are “more likely than not.” Both formulations have been considered, and although the final amendment has yet to be adopted, the drafters see no functional difference between the two.
The amendment’s sponsor asserts that many judges mistakenly admit expert opinions by brushing away evidentiary objections on the ground they go to weight, not admissibility. In giving judicial gatekeepers, before they admit a proposed opinion, an evidence-weighing role that matches the civil burden-of-proof standard, the amendment is supposed to stop that.
Is that a healthy approach? Can the Rule 702 amendment be expected to produce fairer outcomes? If not, why adopt it?
How, for example, is a judge to decide—under a preponderance-of-evidence or more-likely-than-not standard—whether an economist’s opinion rests on “sufficient facts or data” when that economist says that her sample size of 200 incidents is statistically sufficient but the opponent’s economist says that anything fewer than 400 incidents is not? Can a judge simply credit the opposing expert’s position and thus keep the jury from hearing the opinion? What if the difference between 200 and 400 incidents affects only the opinion’s margin of error or its confidence level? What is a judge to do with an opinion that has, say, a 3–7 percent margin of error and an 80 percent confidence level? What if the dueling experts disagree over whether those error margins and confidence levels are acceptable?
Or suppose a real estate appraiser expresses cogent reasons for selecting the specific comparable properties on which the appraisal is based, but the opposing expert, also with cogent reasons, asserts that those properties are too different from the subject property to support a reliable opinion. How in that scenario should a judge decide, under the preponderance or more-likely-than-not standard, that the appraiser’s facts or data are sufficient and that the opinion “reflects a reliable application of the principles and methods to the facts of the case”?
Can a judge fairly decide, under the preponderance or more-likely-than-not standard, that a doctor’s opinion about the standard of care in the relevant community meets the two reliability criteria of Rule 702(c) and (d), and the sufficiency criterion in 702(b), when the opinion is based on the doctor’s experience and training, but the opposing expert, also relying on experience and training, has a contrary opinion?
Historically, those opinion differences are classic jury questions. Under the proposed amendment, however, judges could instead conclude that those questions are first for the court to decide. Even when a proposed opinion is within the discipline’s acceptability range, which the judge might not know, a judge who harbors doubts about the opinion could easily exclude it. Under the amendment, mere uncertainty—which in litigation can easily be manufactured—always favors the opponent.
Turning judicial gatekeepers into evidence weighers will also generate unwanted variability. Here’s why: Rule 702’s legal standard is based on subjective criteria, judgment calls, and matters of opinion—(a) will the proposed expert testimony help the trier of fact; (b) is the testimony based on sufficient facts or data; (c) is the testimony the product of reliable principles and methods; (d) does the opinion reflect a reliable application of the principles and methods to the facts of the case?
When an evidence rule adopts a legal standard based on subjective criteria such as those, a preponderance-of-the-evidence threshold makes admissibility decisions too unpredictable, with case outcomes depending even more on the luck of the judicial draw. Because those decisions would rest on fact-finding and weighing of evidence, reversals would be few, thus making judges less accountable for those decisions. If, as in most cases, the opinion were essential to proving a claim or defense, a decision to exclude it would become a de facto directed verdict, but not under a directed verdict standard.
Any time different judges, each acting reasonably, could reach different conclusions about the same expert opinion, wouldn’t it be better to let the jury hear it? If the jury finds for the opinion’s opponent, no harm will have been done. If the jury finds for the opinion’s proponent, a judge who believed that the Rule 702 criteria had not been met can always consider ordering a new trial or entering judgment as a matter of law, with a reasoned explanation of why the opinion fell short. If the opinion spoke to damages, the judge can propose an additur or remittitur.
Another fault with the amendment is that the four admissibility criteria, while useful to assess scientific opinions, are problematic when assessing opinions based on skill, experience, specialized knowledge, or social science, which Rule 702 also allows. Rule 702’s across-the-board application has been in place since 2000, when the rule was amended to codify Kumho Tire v. Carmichael (1997), in which the Supreme Court, evidently not foreseeing later difficulties, held that the criteria for scientific opinions apply to all opinion testimony.
The proposed amendment magnifies the problem. Consider this example from the 2000 advisory committee notes—an attempt to justify applying the four criteria to nonscientific opinions:
While the terms “principles” and “methods” may convey a certain impression when applied to scientific knowledge, they remain relevant when applied to testimony based on technical or other specialized knowledge.
For example, when a law enforcement agent testifies regarding the use of
code words in a drug transaction, the principle used by the agent is that participants in such transactions regularly use code words to conceal the nature of their activities. The method used by the agent is the application of extensive experience to analyze the meaning of the conversations.
A bit of a stretch, no? People don’t think of an often-observed pattern among drug dealers as a principle or a witness’s use of judgment or experience as a method.
Or look at the example of an experience-based opinion mentioned in Kumho Tire: “a perfume tester able to distinguish among [different] odors at a sniff.” What is the principle that such an expert uses to tell that a particular odor is Chanel No. 5 and not Shalimar? Hard to say. And what is the method, other than “I sniffed it”? Even the Reporter to the Advisory Committee noted in a 2018 memo that experts who “base their opinion solely on experience . . . have never fit all that well within the term ‘reliable methodology.’”
Nor do the Rule 702 criteria work for the so-called knowledge expert, who offers juries contextual or background information about, say, an industry custom and practice, or about how elevators work, or about the safety steps pilots take before takeoff and the reasons for each step.
Because it is hard to apply Rule 702’s principles-and-methods criteria to testimony based on experience or other specialized knowledge, it is no surprise when courts exclude that testimony. When it is hard to discern what is principle or method, it will be even harder to satisfy a judge under a preponderance-of-evidence standard that the principles and methods are reliable and have been reliably applied. The upshot: Judges can be expected to apply levels of rigor that would doom a great deal of nonscientific, yet still jury-worthy, testimony.
A one-size-fits-all rule works only when the items that make up the “all” are alike in all material ways. No one, for example, would expect a university to apply the standard for judging a PhD dissertation in biochemistry to a dissertation on the historical influence of presidential chiefs of staff. Given the breadth and variety of factual disputes and of expertise that can provide valuable help in dispute resolution, it is hard to justify using a preponderance-of-evidence gatekeeping standard, suitable for assessing scientific opinions, to assess opinions based on knowledge, experience, or social science.
We should be cautious about rallying behind this proposed amendment, given the lack of empirical data to justify it. The amendment’s sponsor, relying only on anecdotal accounts, simply says that, “in a fair number of cases,” courts have admitted expert testimony when Rule 702’s requirements have not been sufficiently met. But we haven’t been told how many such cases there are or whether those cases outnumber the cases of wrongful exclusions.
To know whether the rule needs tightening, shouldn’t we see data showing that gatekeeping decisions more often err by admitting unworthy expert testimony than by excluding legitimate expert testimony? Wouldn’t it also help to know the gravity of those errors? For example, when expert opinions have been wrongly admitted, how often was the error harmless? How often was it material to the outcome? And how does that compare with when expert testimony was wrongly excluded? Without the data, how can we have any confidence that the revised Rule 702 won’t be worse than the problem it purports to fix?
In February 2022, the Standing Committee on the Federal Rules received over 500 public comments on the proposed amendment. They can be found at www.regulations.gov/docket/USC-RULES-EV-2021-0005/comments. Unsurprisingly, many lawyers and organizations that defend business interests in civil cases commented to support the amendment, while many plaintiff-side commenters opposed it.
Most of those comments were unhelpful. Laced with what seemed like self-serving pitches about why current gatekeeping practices are either adequate or not, they read as if the commenters were simply saying the amendment should be adopted or rejected because that would make it easier or harder for their side to win. Only rarely did a comment analyze the amendment neutrally from the perspective of sound judicial architecture—what would be fairest for all stakeholders and promote the best interests of justice.
When analyzed from that perspective, a strong argument can be made for a standard that removes judicial evidence-weighing and replaces it with what the legal system traditionally recognizes as objectivity. The admissibility question should not be whether, in the judge’s evaluation under a preponderance-of-evidence or more-likely-than-not standard, the principles and methods used are reliable (yes or no) or the principles and methods have been reliably applied to the case’s facts (yes or no). The question should be whether, if the opinion is based on principles and methods, they and their application are reliable enough—or, conversely, too unreliable—for a reasonable jury to credit the opinion. If an opinion, by its nature, does not depend on principles and methods, the question should be whether the expert’s experience and knowledge of the matter at hand are sufficient for a jury’s consideration.
Under such a standard, if the opinion is relevant under Rule 401 and reliable enough for a reasonable jury to credit it, the judge should allow it into evidence. After that, the opinion’s opponent can attack it through cross-examination, contrary evidence, and arguments of counsel. Couple that with appropriate jury instructions and you have, as the Supreme Court said in Daubert, “the traditional and appropriate means of attacking shaky but admissible evidence.”
Imagine the upgrade in the quality of justice if that were the admissibility standard. Judges would fit their gatekeeping rulings to the circumstances of the case by applying the current Rule 702 criteria to scientific opinions and to parts of nonscientific opinions that rest on facts, data, principles, and methods. But they wouldn’t apply them to opinions that cannot be fairly assessed under those standards. Judges would still exclude junk science, but nonscientific opinions would be admitted or excluded based on their capacity to assist the jury, not on the judge’s evidence-weighing or credibility determinations. Gatekeeping would be more uniform from courtroom to courtroom and more easily reviewable on appeal, thus incentivizing judges to make sure they get it right.
The proposed Rule 702 amendment evolved from the 1990s Daubert and Kumho Tire decisions, which institutionalized a gatekeeping role for judges. Yet, for an amendment that would mandate stricter adherence to facts and data and that strives for reliability, the amendment’s supporters ironically have only anecdotal evidence of allegedly erroneous gatekeeping decisions to justify its adoption.
Without facts and data about unjust outcomes, and with only impressions based on anecdotes, it’s not clear that the amendment will turn judges into better gatekeepers. As some fear, they may just become stricter ones, producing less justice rather than more.
