But verification is not just a part of the nuclear disarmament process—it should be at the root of the process of producing forensic science evidence. Did the fingerprint or firearms examiner get their finding “checked” by a colleague? Indeed, one court deems it so important that an unverified finding was held inadmissible, finding verification “integral to the process. …” People v. Cline, 156 N.E.3d 501, 506 (Ill. App. 1st 2020) (internal quotations omitted).
Why, then, did a judge recently rule that
the State may not present testimony or other evidence that [the firearm examiner’s] conclusions were verified by another firearm’s [sic] examiner. The Court concludes that there is no basis to conclude that the verification process will be helpful to the trier of fact, and there are reasons to question the value of the verifier’s conclusion because the verifier was aware of the prior examiner’s conclusion before performing their analysis.
Order on Pre-trial Motions, State of Oregon v. Benson, No. 22CR57267 (Apr. 18, 2024) (emphasis added).
This is seemingly a first, and to understand it this author turned to Dr. Itiel Dror, the witness who testified as part of the challenge to the firearm examiner’s testimony. I first asked Dr. Dror what the importance of verification is:
The example I like to bring, which I brought up in court: If a physician tells that “you’re going to die in 5 years, unless both of your legs are amputated, and then you will live for 10 years.” You want to verify their conclusion before you consider whether or not to have your legs amputated. … You want somebody else to look at the same medical evidence to see if they reach the same conclusion, to get some verification confirmation, so as to have some confidence in their conclusion.
Now, what is critical in this medical example? If you don’t know, then it is important that when you go to get the second opinion, initially, never tell the second physician what the first physician said, so you can get an honest opinion. Only after they have formed their own opinion independently (blind to the first physician’s opinion) should you share the opinion of the first physician. Not knowing what the first physician said can give a genuine confirmation and verification. If, however, they give you a different opinion, then you can say to the second physician: You know what? Another physician told me something totally different.
… in the forensic examination, supposedly, when they get the verification, a second opinion, they go and say, here’s the evidence. “Here’s what the other physician said. Here’s the name of the other physician. Here’s how they concluded. And why are they correct?” So, you’re not getting the second physician to really engage with the data and evaluate it because they’re already getting the conclusion of the first physician.
As Dr. Dror explained, there are compounding problems.
[T]hat first examiner picks and asks another examiner to verify their work (often it’s not even a supervisor asking someone to verify). I’m examiner one, and I pick who’s going to verify my work, and I come and talk to say, “Jules, can you look and check my conclusion? Was I correct or not?” I verify your work, and you verify my work for decades with colleagues. So, this all dilutes and increases and introduces many, many biases. …
Of concern, of course, is whether having blind verification makes a difference. Dr. Dror’s response drew from experimental findings:
There are data on this, an experiment where you take the same firearm evidence. You do verification that’s blind. You do verification that is not blind. And the agreement in the blind verification is much higher when the verification is not blind. So, this research has been done in firearms. The same evidence and the conclusion was verified blind and not blind, and big differences were found.
In Benson Dror identified another problem of cognitive bias.
The typical example is what happened here, where they go backwards from the reference material. They’re kind of told the theory of who the suspect is, and instead of going from the evidence to see if it fits a suspect, they have the suspect in mind already, and they look to see if the suspect is in the evidence (not to mention knowing that the suspect is black and the victim is white).
These cognitive bias distortions were not enough for the judge to exclude the firearms identification entirely, although such a result was obtained in one case in which Dr. Dror testified in the United Kingdom.
There is an entirely unrelated but equally problematic issue with verification testimony: It is inadmissible hearsay, indeed inadmissible testimonial hearsay that deprives an accused of the right of confrontation.
The potential for hearsay, and, by extension, Confrontation Clause violations, is mitigated when an analyst provides a simple explanation of the ACE-V methodology accompanied by a statement that the methodology was used in a particular case. When, as here, the expert goes further to indicate that the particular identification has been verified by another examiner, problems arise. That is because the import of a statement that the identification has been verified is that the identification has been deemed correct by an expert who reached the same conclusion. Moreover, the value of the verification lies in its truth. The state essentially gets two expert opinions from the testimony of one testifying expert.
State v. Kiser, 2019 Tenn. Crim. App. LEXIS 340, at *25–26 (June 6, 2019). Not all courts agree, with some treating the second opinion as nonhearsay but just as information the first analyst relied on in forming their opinion. Put more simply, Analyst A makes an identification, and then relies on Analyst B’s confirmation to then “finalize” the opinion. “[T]estimony that another examiner had ‘verified’ the work … tended to show that [the testifying examiner] had followed a standard and accepted methodology in her field of expertise, and the verification to which [she] testified, therefore, formed a basis for her own expert opinion.” Jarnigan v. State, 761 S.E.2d 256, 260 (Ga. 2014). The Jarnigan reasoning leaves something to be desired, as what is really occurring is that Analyst A’s opinion was already formed and was just checked; and, of course, there is the problem of explaining to the jury why the second opinion is admissible but not for its truth, a mental gymnastic few can perform.
Where does this leave forensic examiner testimony? Courts should be highly skeptical of nonblind verification: It is bad science and may provide cover for the admission of erroneous findings. And as to the hearsay problem? The court in Cline had an answer: “We find, however, a simple fix: the State should have called the verifying examiner.” 156 N.E.3d at 506.
