February 21, 2013 Articles

Statistical Significance Post-Matrixx

Clearing up some of the confusion caused by the Supreme Court's decision.

David A. Gulley

Tests of the statistical significance of research results are widespread. They are adopted world-wide in a variety of cultures and disciplines that often seem to have little else in common. These tests also make their way into courtrooms. A quick search on Google Scholar yields at least 1,140 federal court opinions that refer to "statistical significance" and 3,040 such opinions that refer to "statistically significant." An overlapping group of at least 2,220 federal court decisions refer to "95 percent." In fact, some of the earliest work in this intellectual vein was undertaken specifically with the justice system in mind; Jacob Bernoulli considered mathematical odds and the reliability of evidence in the context of legal inquiries in the early 18th century. Popular television portrays "forensic scientists" examining patterns and telltale signs in crime-scene evidence, much of which has been subjected to statistical tests in real life.

In a companion academic article, I studied the level of acceptance and rate of adoption of basic tests of statistical significance by the scientific community. D. A. Gulley "The Adoption of Significance Tests by the Scientific Community : An Empirical Analysis" (May 5, 2012). By conducting empirical research on this topic, I wanted to provide some additional objective results to a topic that is usually approached more subjectively. My results may be useful to attorneys in Frye, Daubert, and related hearings on the admissibility of expert evidence. And the Supreme Court's ruling in Matrixx Initiatives, Inc. v. Siracusano, 131 S. Ct. 1309 (2011), makes the subject of statistical significance worth revisiting more generally.

Premium Content For:
  • Litigation Section
Join - Now