Expert witnesses and the lawyers who retain them are well-aware of the dangers of an opinion that strays outside the expert's core area of expertise. Such risks may become particularly acute when the expert embraces methods or research that are inherently controversial and open to challenge. The expert risks exclusion of testimony and the lawyer may wind up having his or her entire case thrown out. A recent decision shows just how real such risks truly are.
In EEOC v. Kaplan Higher Educ. Corp.,748 F.3d 749 (6th Cir. 2014), affirming 2013 U.S. Dist. LEXIS 11722 (N.D. Ohio Jan. 28, 2013), the court of appeals affirmed the exclusion of expert testimony and the dismissal of a discrimination case which challenged an employer's use of credit histories. A key element of the decision was the admissibility of expert testimony.
The EEOC's case involved an employer's use of credit histories. Kaplan made limited use of credit histories in its hiring for certain sensitive positions and race-blind evaluation of the individual circumstances underlying a poor credit history. The EEOC is concerned that poor credit histories may result in a disparate impact against African-Americans in hiring decisions that cannot be explained by factors other than race. However, there appear to be some methodological issues in the research directly supporting this supposition, given the potential for confounding influences, such as income.
In order to bridge the analytical gap between the EEOC's theoretical presumption of disparate impact and the case-specific facts in Kaplan, the EEOC's expert undertook a study purporting to show a statistically significant correlation between rejection rates and race as a result of Kaplan's use of credit histories. To do so, however, the expert needed race data for applicants who were not hired; and these data did not exist. The expert's solution was to have the EEOC subpoena the DMV license records for job applicants from 38 states and to task a team of coders to identify race from a visual inspection of the color ID photographs obtained, where race data was otherwise lacking (24 states).
The trial court found this methodology flawed, leading to a successful Daubert challenge and ultimate to dismissal of the case. The same expert's opinion in Freeman also failed a Daubert challenge, leading to dismissal of that case, as well. In the end, the EEOC could not demonstrate that there was a disparate impact in either company's use of credit histories.
Methodologies based on visual identification have had trouble under Daubert challenges before—most notably in the seminal decision in Kumho Tire. Reliance on visual inspections for determining race has similarly been disfavored by courts before the decision in Kaplan, see Kaplan Higher Learning Educ. Corp., 2013 U.S. Dist. LEXIS 11722, at *25–26; and, as the district court in Kaplan noted, the EEOC, itself, discourages employers from visually identifying an individual by race. Id. at *25. The EEOC's expert in Kaplan tried to overcome objections by using a formalized "coder methodology", a procedure common to certain kinds of academic social science studies. The testifier adopted a highly structured process to ensure inter-coder consistency. But whether inter-coder consistency speaks to coder accuracy, and hence reliability, remained an unanswered question.
The raters were able to reach a consensus on race at the 80% confidence level 88.3% of the time. The team consisted of five individuals with advanced degrees and prior experience involving multi-racial populations, but none had experience in identifying race from photographs. The EEOC's expert could measure "consistency" among the raters, but could not quantify the degree of "accuracy" or margin of error in the coding process. In the end the only support for a claim of "accuracy" which the EEOC was able to offer was anecdotal evidence and the professional judgment of their expert. It wasn't enough. As Judge Blackmun observed in Daubert: "there are important differences between the quest for truth in the courtroom and the quest for truth in the laboratory." 509 U.S. 579, 596–97 (1993).
The EEOC and its expert were therefore rebuked in both the district court and court of appeals: "The EEOC brought this case on the basis of a homemade methodology, crafted by a witness with no particular expertise to craft it, administered by persons with no particular expertise to administer it, tested by no one, and accepted only by the witness himself." Kaplan Higher Educ. Corp., 748 F.3d at 754.
David Gulley is an adjunct professor at Columbia University in New York, New York. Prof. Gulley thanks David B. Ross of Seyfarth Shaw, one of the attorneys who argued for exclusion, and who provided much of this information in correspondence.
Keywords: expert witnesses, litigation, Daubert, testimony, exclusion, admissibility