In the matter of Equal Employment Opportunity Commission (EEOC) v. Freeman, the circuit court ruled on February 20, 2015, to completely exclude the testimony of the EEOC's expert and ruled for summary judgment in favor of Freeman. The suit alleged that Freeman, a company that provides integrated services for expositions, conventions, and corporate events, engaged in a background check process that had a disparate impact on black and male job applicants. In the ruling the circuit court found that Freeman's expert's report "contained a 'plethora' of 'analytical fallacies,' reflected 'cherry-picked' data, produced 'a meaningless, skewed statistic,' and included a 'mind-boggling number of errors.'"
The EEOC hired the expert to review Freeman's applicant employment tracking and background check database to determine if Freeman's background check process adversely impacted black and male applicants. The expert filed the report, and then went on to file three additional amendments to the report, each with different calculations and results. The EEOC claimed that, in one iteration of the amended report, the expert had identified and fixed several errors, but the court found that the amendments were inadequate. The court stated, "Even when Murphy submitted late-in-the-day amendments, he still relied upon 'a skewed database plagued by material fallacies.' The slapdash nature of Murphy's work convinced the district court that the EEOC had only a 'theory in search of facts to support it.'"
The court also found, based upon a comparison of the raw data to the expert's database, that the expert regularly introduced errors into the data and omitted relevant time periods and observations. The court provided three distinct factors that other courts have relied upon when excluding an expert's testimony:
- In Lilly v. Harris-Teeter Supermarket (720 F.2d 326 (4th Cir. 1983)), the court cautioned experts against drawing broad conclusions from incomplete data.
- In Bricklayers & Trowel Trades Int'l Pension Fund v. Credit Suisse Secs. (USA) L.L.C (752 F.3d 82, 92 (1st Cir. 2014)), the court excluded expert testimony that "cherry-picked" relevant data.
- In Brown v. Burlington N. Santa Fe Ry. Co. (765 F.3d 765, 773 (7th Cir. 2014)), the court excluded analysis that contained obvious errors and mistakes.
The court found that each of these three factors were applicable to Freeman's expert's report in this matter.
While this case appears to present an extreme example of unacceptable expert work, the ruling is a reminder to all experts to be vigilant in proposing only sound, intellectually honest methodologies, supported by careful data handling techniques and processes designed to minimize, to the extent possible, data and calculation errors. Experts reviewing data are often tasked with determining the validity of the data received and must make choices to include or exclude certain pieces of data. It is accepted good practice in data analysis to sometimes exclude outliers or exclude other data anomalies. Excluding certain data is considered acceptable—and necessary—when there is good, disciplined reason to believe that including that data would genuinely create misleading results or cause statistical fallacies. The court points out that excluding data in order to support a pre-conceived theory is not acceptable, however.
The ruling is also a reminder to attorneys to ensure that they understand their expert's methodology, that the methodology meets or exceeds litigation standards, and that data handling and data editing techniques are consistent with accepted good practice.
Click here for more information.
Jeremy Guinta is an associate director at Navigant Consulting in Los Angeles, California.
Keywords: expert witnesses, litigation, EEOC, Freeman, errors, mistakes, inaccurate report