chevron-down Created with Sketch Beta.

    The Judges' Journal

    Technology Part 2

    Beyond the Mysterious: Forensic Science Standards and the Law

    Dana Delger and John Paul Jones II

    Jun 11, 2024

    16 min read

    Access to Justice Court Technology Courts & Practice Areas Fairness & Impartiality Federal General Practice Improving the Courts Rule of Law Technology

    Summary

    • A basic introduction for judges to the important world of forensic science standards.
    • The National Institute of Standards and Technology’s work in forensic science standards, through the Organization of Scientific Area Committees and other efforts, can help foster not only better science within labs but also a more fulsome understanding of science in courtrooms.
    • Human Factors Task Group expertise is used to help the various subcommittees draft standards that appropriately address the wide range of human factors that can affect the outcome of forensic examinations.
    Beyond the Mysterious: Forensic Science Standards and the Law
    MirageC via Getty Images

    Jump to:

    Any sufficiently advanced technology is indistinguishable from magic.

    —Arthur C. Clarke

    When science fiction writer Arthur C. Clarke wrote these words in 1973, he likely was not thinking of forensic science. Forensics, however, is complex, and the results it produces can often appear to be magic, rather than science, to nonscientists in the legal system. This may be especially true where, as is increasingly the case, courts are faced with new technologies, some of which change how traditional forensic techniques are practiced and others offer wholly new kinds of evidence. But unlike science fiction readers, the law cannot afford to be blind to how evidence is collected and scientifically analyzed to produce results. Whether a court approaches these technologies through the gatekeeping lens of Federal Rule of Evidence 702/Daubert v. Merrell Dow Pharmaceuticals, Inc. (and their state equivalents), or asks instead whether a technique has been generally accepted by the relevant scientific community under Frye v. United States, the fair administration of justice requires the judiciary to move beyond the mysterious into scientific understanding.

    An understanding and application of forensic science documentary standards are means by which courts can help make this shift. These documents, which guide the practice of forensic techniques, themselves play a number of important roles in the legal system. When raised in litigation, for example, standards can be a key part of admissibility determinations. Daubert notably looks to “the existence and maintenance of standards controlling [a] technique’s operation” as one factor courts can consider in determining the admissibility of scientific evidence, and courts in Frye jurisdictions have also looked to standards as evidence of general acceptance. But the utility of standards goes beyond direct applicability in court, and judges should not wait for litigants to raise them to become educated about what they are and what they mean.

    As one 2021 survey of judges found, “further judicial training [on forensic science] is important not because judges are unaware of differences in the reliability of forensic evidence, but rather because judges would like to render more informed rulings on these technical subjects.” Currently, such judicial training in the area of standards is lacking. The same survey found that only 27.7 percent of judges had received training in “standards and terminology for expressing conclusions,” and there is no reason to think judges receive substantially more or better training on other kinds of standards. Indeed, although there may be real, practical limitations to what judges can directly do with standards when they have not been raised by litigants, an understanding of forensic science standards and the process by which they are developed can still be of tremendous benefit to the legal system. As the survey found, “[n]early all judges reported that they should receive training on forensic science evidence through continuing education as a judge.” Thus, even where standards cannot directly form a basis for decision-making, judges who have a facility with forensic science standards will still be better equipped to understand and manage forensic evidence and, ultimately, render the informed rulings any court aims to.

    This article aims, then, to begin to fill that gap and provide a basic introduction for judges to the important world of forensic science standards. Currently, most of the forensic science standards activities in the United States are being facilitated by a program in the National Institute of Standards and Technology (NIST), a nonregulatory bureau of the U.S. Department of Commerce, which has the mission to “promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life.” Since 2014, NIST has operated the Organization of Scientific Area Committees (OSAC) for Forensic Science with the mission to “strengthen the nation’s use of forensic science by facilitating the development and promoting the use of high-quality, technically sound standards.”

    Taking as its lens standards specific to digital evidence, i.e., the “field of forensic science that is concerned with retrieving, storing and analyzing electronic data that can be useful in criminal investigations,” this article will introduce OSAC and its outputs, starting with a description of OSAC’s structure and how it facilitates the standards development process. It will then examine one of the parent scientific area committees, the Digital/Multimedia Scientific Area Committee, and its four subcommittees—Digital Evidence, Facial & Iris Identification, Speaker Recognition, and Video/Imaging Technology & Analysis (VITAL)—and look at some example standards in each area. Each of these disciplines is at the forefront of the technology being used in criminal investigations and trials, and awareness of the standards each subcommittee produces is an important means by which courts will not be left behind as technology advances. Finally, the article describes some of NIST’s important work addressing human factors, which are important even in the digital space. Judges should leave this article with a better understanding of what standards are and how they are produced so that regardless of whether standards are directly deployed in their courts, they will be in a better position to render well-informed rulings.

    OSAC Background and Structure

    A “documentary standard” at its basic level is a document that describes an agreed-on way of doing something. When you add the terms “voluntary” and “consensus” to the term “standard,” you have now signaled that a document has undergone a process where a group of individuals have discussed the topic covered by the standard, generally agreed with its content, and made the document available for voluntary use by relevant communities. In the United States, the federal government is encouraged to collaborate with private-sector standards developing organizations (SDOs) on the production of voluntary consensus standards, which are ultimately published by those private organizations, instead of creating government-unique standards. OSAC is a federal program designed to facilitate the development of new voluntary consensus standards where needed and perform in-depth reviews of existing published standards to determine their technical merit. The goal of these activities is to populate the OSAC Registry, a public repository of voluntary consensus standards recommended for use by the forensic science community, with high-quality forensic science standards that have been vetted by a diverse group of stakeholders. OSAC uses the Registry as a signaling tool to highlight specific standards that have completed the technical review process and encourage the forensic science community to implement these standards into their operations.

    OSAC, while administered by a small team of federal staff, leverages over 800 volunteer experts to execute its activities. OSAC has a hierarchal structure that was developed to ensure a designated focus point for each forensic science discipline (e.g., DNA and firearms). These disciplines are grouped into specific scientific concept areas (e.g., biology and physics and pattern interpretation). All these units are governed by the Forensic Science Standards Board (FSSB), a 20-member group of diverse stakeholders from the forensic science ecosystem. Each of OSAC’s 22 discipline-specific subcommittees, noted in Figure 1, is populated with forensic scientists from that discipline and other relevant stakeholders, including academic researchers, legal representatives, statisticians, and quality management and human factors experts. These perspectives enable the subcommittees to engage in robust conversations about forensic science techniques, training requirements, interpretation/opinion scales, terminology, and many other topics that benefit from standardization. Subcommittees that share similar scientific principles are grouped in the same scientific area committees (SACs) to enable collaboration. OSAC also has resource task groups that provide feedback and advice related to human factors, legal, quality, and statistics topics to the subcommittees to enhance the utility of their documents.

    Documents drafted within OSAC are called OSAC Proposed Standards; once they complete the OSAC review process, which includes two-thirds consensus approval by the subcommittee and two-thirds consensus approval by the FSSB, they are placed on the OSAC Registry. OSAC Proposed Standards are then sent to an SDO, where they will complete the SDO’s formal consensus process and eventually be published. It should be noted that draft standards going through the SDO’s consensus process may be changed as SDO participants provide additional input. After a standard is published by the SDO, it will be reviewed by OSAC to ensure it is still of high quality and approved to replace the OSAC Proposed Standard on the OSAC Registry. Because it can take several years for a standard to be developed through the SDO process, OSAC promotes the adoption of the OSAC Proposed Standard while the document works its way through the formal SDO consensus process. The OSAC Registry currently has 182 standards (126 SDO published and 56 OSAC Proposed covering over 20 forensic science disciplines) recommended for adoption by forensic science service providers (FSSPs). These FSSPs include both the larger public crime laboratories that routinely provide forensic science reports in criminal cases and also the smaller digital evidence, crime scene investigation, and fingerprint units that are mainly housed in law enforcement agencies.

    Standards are only beneficial if they are used, so OSAC also tracks the implementation of the standards listed on the OSAC Registry. To date, 152 FSSPs have indicated to OSAC that they have implemented standards listed on the OSAC Registry, and many more are believed to have implemented these standards as well. OSAC is having a significant impact on forensic practice, one that will continue to grow as more standards are added to the OSAC Registry and as more FSSPs incorporate these documents into their operations. These documents are important resources, not only for the FSSPs who use them but also for the judiciary and counsel to enhance their understanding of various forensic science disciplines and the community’s current recommended practices.

    Figure 1: OSAC Organizational Structure

    Figure 1: OSAC Organizational Structure

    Digital Evidence

    OSAC’s Digital Evidence Subcommittee focuses on standards and guidelines related to information of probative value that is stored or transmitted in binary form. As of October 2023, this subcommittee has had five standards placed on the OSAC Registry. Of these standards, three have been developed through ASTM International, an SDO with engagement in many industries that, prior to its involvement in forensics, has a long history of acceptance by the legal system. Two other standards have been developed through the Scientific Working Group on Digital Evidence (SWGDE), which currently operates as an SDO for digital evidence.

    ASTM E3017-19, Standard Practice for Examining Magnetic Card Readers, is a good example of a digital standard for this subcommittee and illustrates some of the types of information included in standards that can aid courts in dealing with technology. This standard lays out how examiners should seize, acquire, and analyze data from magnetic card readers, known as skimmers when used unlawfully. These examinations can reveal (or equally fail to reveal) evidence of “unauthorized” (i.e., stolen) information and so can play an important role in defending or prosecuting criminal cases.

    Within this standard, courts can find explanations of important terminology, information about collection techniques and data analysis, and some discussion of the technique’s limitations, among others, all of which can aid courts when considering the examination of skimming devices. As the standard itself states, its application “cannot replace knowledge, skills, or abilities acquired through education, training, and experience.” This standard, and others developed and promoted by this subcommittee, can give judges a place to begin their understanding of digital evidence and where and how it fits into the proceedings in their courtroom. For example, a court in a jurisdiction where standards on the OSAC Registry have been implemented might expect to hear testimony about this standard when skimming cases are heard; in those circumstances, a judge deciding admissibility (or sitting as a fact-finder) might consider how the evidence being offered comports with, or fails to comport with, the standard. Even where this standard is not raised by litigants or is otherwise unavailable to be considered directly by the court, a preexisting familiarity with the standard would nonetheless aid the court in understanding the evidence being offered.

    Facial & Iris Identification

    OSAC’s Facial & Iris Identification Subcommittee focuses on standards and guidelines related to the image-based comparisons of human facial and iris features. As of October 2023, this subcommittee has nine standards on the OSAC Registry, including a joint standard detailed below. Five of those standards are OSAC Proposed Standards, which are also currently with ASTM International and working through the ASTM standards development process. While facial identification is often highly technical, the standards currently on the OSAC Registry reflect the broader scope of facial identification practice, including facial comparison, which is “a manual process conducted by a human which entails identifying similarities and differences between two images or an image and a live subject to determine whether they represent the same person.”

    One important OSAC Proposed Standard that is currently under development at ASTM International is OSAC 2022-S-0007, Standard Guide for Facial Comparison: Overview and Methodology Guidelines. While this OSAC Proposed Standard is being finalized at ASTM and is still subject to change, OSAC encourages its adoption while the SDO process continues. Our belief is that OSAC Proposed Standards that have completed the development and review process at the OSAC level are sound interim documents that should be available to and considered for implementation by the forensic science community.

    ANSI/ASTM E3149-18, Standard Guide for Facial Image Comparison Feature List for Morphological Analysis, currently on the OSAC Registry, details a “set of facial components, characteristics, and descriptors” that should be used during facial comparisons using morphological analysis, the recommended method for manual facial comparison. This standard, read together with OSAC 2022-S-0007, Standard Guide for Facial Comparison: Overview and Methodology Guidelines, represents the ongoing nature of standards going through OSAC’s Registry approval process. No subcommittee has or will have “complete standards” that cover every aspect of a forensic science discipline. Not only does the science on which digital evidence rests constantly grow and change, but as OSAC’s work continues, these standards will build on each other over time.

    In addition to these standards, two more of the Facial & Iris Identification Subcommittee’s standards deal with how facial images for comparison should be taken for both living and deceased subjects.

    Speaker Recognition

    OSAC’s Speaker Recognition Subcommittee focuses on standards and guidelines related to the practice of speaker recognition, voice data collection, measurement, transmission, and retrieval. The Speaker Recognition Subcommittee currently has one standard on the OSAC Registry, AES76-2022: AES Standard for Audio Forensics—Speech Collection Guidelines for Speaker Recognition: Interviewing at a Temporary Location. This document explains how known speaker samples to be used in a later speaker-recognition process should be collected in nonlaboratory locations. This subcommittee also has a number of important documents under development, including a validation standard for forensic speaker recognition.

    Outside of its standards drafting efforts, the Speaker Recognition Subcommittee has also produced a “process map,” which is a graphical representation of the discipline’s current practices. This process map shows in detail the decisions examiners in this field make at different junctures and some of the varying paths examinations might take. This document can help courts understand the current approaches in the speaker recognition discipline.

    Video/Imaging Technology & Analysis (VITAL) Subcommittee

    OSAC’s VITAL Subcommittee works on standards and guidelines related to the application of methods and technologies to analyze information related to forensic imagery from a variety of systems. The impact this discipline has across the forensic space is seen in the variety of its documents, both in the SDO published and OSAC Proposed Standards that are on the OSAC Registry. For example, one VITAL standard on the OSAC Registry is ANSI/ASTM E3235-21, Standard Practice for Latent Print Evidence Imaging Resolution. This standard provides recommendations on the level of resolution necessary for latent print imagining systems and equipment to enable fingerprint examiners to make appropriate comparisons. Such a cross-disciplinary effort reveals the importance of technology, not only to digital forensics but also to traditional forensic techniques. VITAL has also shared its work on standards with the Facial & Iris Identification Subcommittee. Together, these subcommittees have collaborated on several OSAC Proposed Standards listed on the OSAC Registry—one addressing post-mortem photography and another entitled OSAC 2022-S-0001, Standard Guide for Image Comparison Opinions (see below).

    Interdisciplinary Standards

    An advantage of OSAC’s structure and its use of various SDOs is that, in addition to discipline-specific standards, standards that span multiple disciplines can be drafted and placed on the OSAC Registry. These interdisciplinary standards can be of importance in rapidly changing technological environments. One such standard is ASTM E2916-19, Standard Terminology for Digital and Multimedia Evidence Examination, which “is a compilation of terms and corresponding definitions used in the examination of digital and multimedia evidence to include the areas of computer forensics, image analysis, video analysis, forensic audio, and facial identification.” An interdisciplinary standard such as this can help ensure that experts and courts are speaking (and understanding) the same language.

    Another critical interdisciplinary standard is OSAC 2022-S-0001, Standard Guide for Image Comparison Opinions, which involves collaborations between members of OSAC’s VITAL and Facial & Iris Identification Subcommittees. This OSAC Proposed Standard, currently included on the OSAC Registry and at ASTM International for further development, aims to “increase harmonization and consistency by providing a framework of opinion categories across and within the digital multimedia forensic disciplines that compare images.” The OSAC Proposed Standard directs disciplines involved in image comparisons to use this standard when developing and validating discipline-specific opinion categories and to reference “any empirical studies of the scale for opinions about the weight of evidence for the applicable type and quality of evidence.” Importantly, the standard states that, currently, few “standardized opinion scales with associated validation data . . . exist.” The standard goes on to require opinion scales produced to this standard to “explicitly state” when that is the case, i.e., when no relevant validity studies exist. This standard is of great value to a judiciary and a legal system trying to grapple with how to appropriately understand and weigh forensic evidence.

    Other Resources

    Several other materials exist that can aid courts in understanding the standards of the OSAC Registry. Under a cooperative agreement with NIST, the American Academy of Forensic Sciences (AAFS) has produced fact sheets, which are one-page documents intended to summarize standards in clear and easy-to-understand language. Fact sheets are available for most standards on the Registry and can be accessed by a hyperlink that is under the standard on the OSAC Registry webpage. These documents can serve as a useful launch point for courts in understanding what a standard means and how it fits into a broader forensic practice. It should be noted that these fact sheets are not intended to serve as definitive interpretations of the standards and are not produced by NIST.

    Courts in the process of undertaking general education about standards may also be interested in the list of available research and development (R&D) needs produced by each subcommittee. These R&D needs describe specific gaps in the knowledge base and can be used to support the drafting and refinement of standards as well as to draw in research to help fill those gaps to provide a stronger scientific foundation. These assessments also include bibliographic references and explanations for why this research is necessary. For example, in 2021, the VITAL Subcommittee’s Research Needs Assessment for Detecting Deepfakes identified that there was a major gap in the current knowledge and that there was no or limited research being conducted on this important problem, which could, per the subcommittee, result in “geopolitical, criminal implications, or civil unrest.” Since its posting, there has been significant research performed in this area, both in forensic science and in the broader scientific community. These documents illustrate what some of the most experienced and active practitioners in a field view as the state of their science and their efforts to steer researchers into working on these important areas to assist with these challenges, and, to that end, can serve to educate judges about where research may be most needed in particular fields.

    Other Technological Standards

    Technological advances in forensic science are not limited to digital evidence, and the standards on the OSAC Registry reflect that. For example, DNA analysis has undergone significant changes since its first usage, and now many laboratories and FSSPs use computer-based probabilistic genotyping systems to “infer genotypes and/or calculate likelihood ratios.” One important standard on the OSAC Registry is ANSI/ASB Standard 018, Standard for Validation of Probabilistic Genotyping Systems, which lays out what a practitioner must do to validate their lab’s system. The future technologies being brought online to support the examination of firearms are also addressed with standards. There are three standards on the OSAC Registry that provide guidance for implementing new 3D measurement systems for enhancing the traditional 2D method of firearms analysis and comparisons. Over time, these standards will help guide laboratories on how to implement these new technologies into their operations following a technically sound approach. As technology continues to shape forensic practice, so, too, will the standards on the OSAC Registry reflect those changes in various disciplines.

    A Note on Human Factors Both in OSAC and Beyond

    Even the most technologically advanced forensic technique is, at its core, a human endeavor and can be subject to error. The structure of OSAC implicitly recognizes this, and one of the task groups that serves as a resource to the various committees is the Human Factors Task Group. Human factors broadly “examine[ ] the interactions between humans and other elements of a system—technology, training, decisions, products, procedures, workspaces, and the overall environment—with the goal of improving both human and system performance.” Accordingly, the Human Factors Task Group is made up of people chosen for their “experience in psychology, cognitive science or a related social science discipline, and knowledge of social science literature on human judgment, decision making, observer effects, communication and cognitive bias.” This expertise is used to help the various subcommittees draft standards that appropriately address the wide range of human factors that can affect the outcome of forensic examinations.

    Outside of OSAC, NIST has been engaged in other efforts to promote a fuller understanding of human factors in forensic analysis. For example, NIST has published two Expert Working Group Reports on human factors topics—one on latent fingerprint examination and one on handwriting analysis. A third report is forthcoming on human factors in DNA analysis. These reports represent an important contribution to the scientific understanding of human factors in forensic science, and courts may find them a useful entry point into this field of study.

    Conclusion

    Forensic science and the technologies that support it can be intimidating, but it does not have to be. NIST’s work in forensic science standards, through OSAC and other efforts, can help foster not only better science within labs but also a more fulsome understanding of science in courtrooms. OSAC’s work can be an important guide to the judiciary in its efforts to ensure its use of science is consistent with the goals of justice and fairness for those who come before it.

      Authors