Criminal Justice Section  

    Criminal Justice Magazine

Criminal Justice Magazine
Summer 2000
Vol. 15, Issue 2

1ex·pert \ 'ek,sp rt, . . . adj [. . . fr. L expertus, past part. of experiri to try-more at 1Experience]

1ex·pe·ri·ence \ ik'spire- n (t)s, . . . n [. . . fr. L experientia, fr. experient-, experiens (pres. part. of experiri to try, . . . ]

(Webster's Third New International Dictionary Unabridged 800 (1961))

As the common Latin root of "expert" and "experience" suggests, the Supreme Court quite sensibly held in Kumho Tire Co. v. Carmichael, 119 S. Ct. 1167 (1999), that a trial judge's gatekeeping obligation under Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), to ensure the reliability and relevance of expert testimony under Federal Rule of Evidence 702 applies to all forms of expertise, including specialized knowledge based on experience. This article focuses on an issue that will now recur in complex criminal cases such as health care and securities fraud as well as narcotics prosecutions: How to test the reliability of experience-based opinions on such topics as common industry knowledge, relevant business practices, and modus operandi to determine their admissibility under Rule 702.

The stakes for both sides are substantial. In a fraud or narcotics case, for example, the admission of such evidence under Rule 702 anchors the prosecution's arguments about the defendant's guilty knowledge and intent or abusive billing, reporting, or accounting practices. And qualifying such testimony as "expert" opens the door to hypothetical questions and the use of inadmissible hearsay before the jury. Conversely, the defense wins a huge victory by excluding such expertise altogether or even shoehorning the testimony into the sphere of lay opinion under Rule 701, which would insulate the jury from hypotheticals, hearsay, and the appellation "expert" being affixed to the witness.

Admission of expert testimony before Kumho

Expert evidence about billing, reporting, and accounting practices as well as industry knowledge and custom is routine in complex fraud cases. In United States v. Erickson, 75 F.3d 470, 475-76 (9th Cir.), cert. denied, 517 U.S. 1222 (1996), the court allowed a number of physicians to testify in a health care fraud case about their common understanding of Medicare billing codes. In a bank fraud case, United States v. Riddle, 103 F.3d 423, 430 (5th Cir. 1997), the court approved the use of a defense expert to discuss how banks operate. Although not a criminal case, in Western Indus., Inc. v. Newcor Canada Ltd., 739 F.2d 1201-02 (7th Cir. 1984), the court permitted expert testimony on trade custom and usage.

Of course, expert testimony by federal agents about modus operandi has long been a staple of drug prosecutions, covering topics such as organizational structure, countersurveillance, and code words, e.g., United States v. Watson, 171 F.3d 695, 703 (D.C. Cir. 1999). Prosecutors now use agents as modus operandi experts more regularly in fraud and other financial cases to explain how a boiler room operates or a check kite works. ( United States v. Whitehead, 176 F.3d 1030, 1035 (8th Cir. 1999) (check kiting); United States v. Yoon, 128 F.3d 515, 527-28 (7th Cir. 1997) (check kiting); United States v. Oriea, 29 F.3d 185, 189 (5th Cir. 1994) (structuring); United States v. McCollum, 802 F.3d 344, 345 (9th Cir. 1986) (mail fraud scheme); United States v. Johnson, 735 F.2d 1200, 1202 (9th Cir. 1984) (securities fraud).)

For example, in a health care fraud prosecution, expert testimony may be used in a couple of ways to prove fraudulent intent and the existence of a scheme. When a physician or hospital is accused of falsely billing Medicare, the government would likely offer expert testimony on the common understanding and use of billing codes and cost reports to discredit the defendant's practices-because both billing codes and cost reports represent the key information relied on by Medicare in calculating reimbursement amounts. The defense should counter with its own billing expert. The defense should also expect an experienced FBI agent to describe the anatomy, or modus operandi, of a fraudulent billing scheme, which usually tracks the defendant's conduct. The agent would describe how a typical physician's office is structured, who creates the list of procedures allegedly performed on each patient, how that list gets converted into a bill, and how the bill is ultimately inflated. Of course, lacking access to the raw investigative data available only to the government and any reasonable means to gather comparable information, the defense would not be able to present an anti-modus operandi witness. But the common denominator for each of these expert's opinion is specialized knowledge grounded in experience.

Since 1993, the battle over admitting experience-based expert testimony concerned simply whether Daubert's gatekeeping inquiry applied to experience-based opinions. In United States v. Webb, 115 F.3d 711, 716 (9th Cir.), cert. denied, 118 S. Ct. 429 (1997), the court said Daubert did not apply to specialized knowledge testimony on modus operandi. In United States v. Jones, 107 F.3d 1147, 1157-58 (6th Cir.), cert. denied, 117 S. Ct. 2527 (1997), Daubert was not applied to handwriting analysis. In United States v. Muldow, 19 F.3d 1332, 1338 (10th Cir. 1994), modus operandi testimony was not subjected to Daubert.

On the other side was United States v. Yoon, 128 F.3d 515, 527 (7th Cir. 1997), in which the court held that Daubert applied to specialized knowledge of modus operandi for check kiting. And in Brinati v. KPMG Peat Marwick, 2 F.3d 183, 186-87 (7th Cir. 1993), the court again extended Daubert to an accountant's experience-based testimony about generally accepted accounting standards.

Nevertheless, the general rule for agents and similar experts was that Daubert did not apply. Rather than scrutinize the reliability of the witness's testimony, the courts instead equated outstanding qualifications with trustworthy opinions. ( See, e.g., United States v. Webb, 115 F.3d 711, 713-14 (9th Cir.) (allowing an expert opinion about how firearms are concealed in a prosecution for possession of ammunition by a felon), cert. denied, 118 S. Ct. 429 (1997).) In fact, one appeals court said that if the gatekeeping rules were applied, "many types of relevant and reliable expert testimony-derived substantially from practical experience-would be excluded." ( United States v. Jones, 107 F.3d 1147, 1158 (6th Cir. 1997).)

Kumho has settled that issue and rightfully raised the bar for any party seeking to use experience-based opinions. No longer is expertise a proxy for reliability. Simply cataloging the expert's experiences, education, and training-regardless of what kind or how much-will not pass muster under Kumho. ( Compare Ambrosini v. Labarraque, 101 F.3d 129, 140 (D.C. Cir. 1996) (finding an expert's testimony reliable in no small part because of his outstanding qualifications).)

If the approach were otherwise, Rule 702 would be turned on its head with expert testimony grounded in the methods of hard science being scrutinized far more critically than more elusive nonscientific opinions. Now, the trial court must assure itself that the expert's experience-based opinion is well reasoned, consistent with the generally accepted principles used by the expert outside of court, and not merely subjective or speculative.

Post- Kumho reliability

The question, then, is what factors a trial court should use to assess the reliability of experience-based testimony. No court has squarely addressed the issue, and the Supreme Court has offered little guidance. In fact, Kumho holds that because Daubert's four nonexhaustive factors for assessing reliability-testability, rate of error, peer review, and general acceptance-are neither exclusive nor definitive, they may be of no use in gauging the reliability of expertise on such topics as modus operandi, code words, or attorneys' fees. ( Kumho, 119 S. Ct. at 1175-76.)

The example of the health care provider highlights the factors' general unsuitability in fraud cases. Neither skilled industry professionals nor law enforcement, in search of immutable theorems of human behavior, are in the business of doing controlled experiments to test hypotheses about innovative billing code applications, new industry customs, or the inherent characteristics of fraud schemes. The kind of statistically random information that girds controlled scientific experiments is, for all practical purposes, unavailable in a fraud context for reasons that include identification, verification, and the privilege against self-incrimination. Instead, collective experience from live, uncontrolled situations provides the grist for business practices and modus operandi testimony. With untestable hypotheses and nonexistent theorems, talking about the error rates of experience is senseless. Nor is peer review and publication applicable to the investigative experiences of federal agents, although those variables may help gauge opinions on industry practices. On the other hand, the factor of general acceptance may be useful in testing the reliability of how skilled professionals and agents collect and apply their experiences.

Realizing the Daubert factors would not work in many nonscientific situations, Kumho granted trial judges "considerable leeway" in framing the reliability analysis, saying-not so helpfully-that each case's circumstances will guide the trial court's formulation of its reliability test and that the Daubert factors should be considered "where they are reasonable measures of the reliability of expert testimony." ( Id. at 1176.)

Although a trial judge may know the cases in which to apply those factors "when he sees them," Kumho does little to tip off attorneys as to when that might be or what to argue in lieu of the Daubert factors. Ironically, in a case about plumbing the reliability of other people's opinions, Kumho allows judges to employ their gatekeeping expertise for experience-based testimony through methodologies that have never been disclosed or tested. In all probability, they are untestable, lack controls of any kind short of appeal, have no history of accuracy or meaningful error rates, have never been subjected to peer review, and know no general acceptance among the courts. If put on trial themselves, such methods would likely be treated as the sort of "junk science" judicially condemned by Daubert.

Kumho, however, did offer one hint for assessing expertise based on experience, such as industry custom and knowledge and modus operandi testimony. The Court said that in the case of a perfume tester-whose opinion, like that of an industry expert or agent, is based almost entirely on experience-it may be useful to ask whether his preparation is of a kind that others in the field would recognize as acceptable. ( Id.)

Factors that measure experience

A similarly fundamental question should be put to experts on custom and usage, billing specialists, agents, and all whose opinions spring from experience. They must be asked if the process for gathering the data underlying the opinion-including education and training-was performed by them or others in a manner acceptable in the expert's field. This will ensure the expert has tied his opinion to reliably obtained information, at least in the view of experts in that field. By so doing, the witness shows his testimony is more likely the product of objective observation than subjective or speculative information. For example, if an agent will be testifying about the mode of operation of a narcotics organization or a hospital, the agent should demonstrate that the information on which his opinion is based-what amounts to other case histories-was gathered by him and others in a manner relied on by law enforcement organizations generally.

Of course, if the data collection process has been badly managed from a statistician's perspective, that may be a valid argument against reliability and, hence, admissibility. For example, data gathering is badly managed when it draws from a biased, numerically insufficient, or underrepresentative population or is premised on incorrect probabilities. Because the essential point of testimony about industry custom and knowledge and modus operandi is an inference based on the sample population about what one of the parties knew or should have known, or how the party acted, the selection and evaluation of that sample population is critical. It may be that the expert's sample population fails to account for similar conduct or is overly inclusive and, thus, too generalized. This kind of complaint, however, may go only to the opinion's weight before the jury, not admissibility, as the Advisory Committee notes to Rule 702 expressly contemplate purely experiential expertise.

Three related factors a court should consider are, first, the number of experiences from which the expert's opinion is drawn; second, the similarity of the experiences themselves; and, third, the resemblance between those experiences and the facts of the case. Consider, for example, an industry expert's experience-based custom and usage testimony about Medicare billing codes. Critical to the reliability assessment is not only the expert's general familiarity with the code but the context in which the expert used the code; as well as how many times and in what manner the expert has used the code; how many times and in what manner others have used it; and the relationship between this data and the facts of the case under indictment. Thus, to guard against the threat of anecdotal information that might unfairly skew the opinion-making process, the number of experiences generally should approach a statistically meaningful level. Likewise, requiring the underlying experiences to be sufficiently similar creates a population of experiences from which the expert can draw reliable conclusions about common characteristics and inferences about allegedly similar conduct. Finally, assessing whether the expert's bank of experiences resembles the facts of the cases helps protect against a subjective or speculative opinion that is not moored to the data.

Another important factor to weigh is whether the expert reached his or her opinion through the same reasoning process, or methodology, that would be used outside of court. Regardless of the kind of expertise, it will be important to know if the witness has reached different conclusions on similar facts when not in court. The number of such instances should inversely relate to the expert's reliability, absent a sound explanation. Perhaps the clearest example is modus operandi testimony, when the judge should be made aware of cases similar to the defendant's in which the agent's methodology has led to specific investigations not being pursued or the nonprosecution of specific subjects after an investigation. Thus, in the narcotics context, if a prosecution of an alleged courier or lookout has been declined, then the reason for that decision certainly should bear on the reliability of the agent's opinion in a like case.

Finally, experts must explain how their experiences led to the conclusions reached and whether they were accomplished in a manner generally accepted in their field. Such a canvass requires the witness to articulate each of the inferential links-the actual thought process-connecting the underlying data to the opinion. This is the very type of testimony Fed. R. Evid. 701 excuses when a lay witness opines on a matter of common experience. On the other hand, given the need for reliable conclusions about complex matters, Rule 702 experts must be able to verbally construct the bridge from their experiences to their opinions using a generally accepted method. United States v. Cortez, 449 U.S. 411, 417 (1981), provides a good analogy for how the witness should frame the analysis. In Cortez, the Court held that in deciding a challenge to an agent's finding of particularized suspicion, the relevant factors must be seen through the eyes of those versed in the field of law enforcement. Cortez links the agent's decision-making process to an objective, or generally accepted, assessment of the information.

These six factors offer a comprehensive benchmark for assessing whether an expert's experiences suggest reliable testimony. Of course, each factor's relative weight may vary in relation to the facts of the case. Some cases also may require the use of additional factors.

Applying the factors at trial

The health care provider example offers a glimpse at how the six factors may work at trial. With regard to an agent opining on modus operandi, the prosecution must show that the supporting experiences, whether personal or collective, derive from accepted law enforcement methods, such as undercover operations; consensually taped conversations; testimony or documented debriefings of offenders, confederates, unwitting accomplices, and industry personnel; and documentary analysis.

The raw number of investigative experiences should be substantial. They should cover similar fraudulent conduct by health care providers. Those frauds should resemble the health care scheme charged in the indictment.

The agent must explain how his evaluative approach to forming his opinion about the alleged health care scheme-identification of telltale signs versus red herring clues and the weight accorded to each-comports with his or her assessment in the field of whether to investigate or recommend the prosecution of a health care provider. The agent should also offer an analysis of cases declined for investigation and prosecution, using the evaluative approach underlying his or her proposed testimony.

Finally, the agent must detail how the investigative data produced the telltale signs, the red herring clues, and the weights ascribed to each. There must be an explanation of the agent's vetting process-which itself must be a generally accepted one-in which schemes were reconstructed and compared, whether by the agent or others. The expert should state the frequency of leading indicators, thereby explaining their relative significance. Based on this kind of testimony, the agent may then reliably opine that the conduct of the defendant health care provider was fraudulent.


The reliability analysis Kumho now requires for experience-based expert testimony offers lawyers a wide avenue for creativity. Although a number of proposed changes to Rules 701, 702, and 703 dot the horizon, none of them directly addresses the very thorny issue of how to evaluate the reliability of specialized knowledge grounded in experience. One argument seems clear, however. Reliability is all about "process, process, process."

Marc N. Garber is an assistant U.S. attorney for the District of Nevada. The views expressed in this article are exclusively those of the author and not the U.S. Department of Justice. A shorter version of this article appeared in the White Collar Crime Reporter 's March 2000 issue.