Whether the Theory Has Been Tested
Can forensic document examiners actually perform the duties of establishing authorship? Two independent series of studies have shown with no doubt that properly trained forensic document examiners are statistically better at authorship discernment than equally educated control groups. Trained forensic document examiners consistently outperformed their control groups by an average factor of six or more. Because the stated purpose of expert testimony (per Federal Rule of Evidence 702) is to provide assistance to the trier of fact, it needs to be understood that the trier of fact is tantamount to the control group—well educated but not specifically trained in forensic science. As such, forensic document examiners can assist the triers of fact to better understand and weigh the importance of documentation evidence through their opinions and supporting data. Based on the above studies, judges should understand that juries would be six times more likely to misinterpret evidence without expert guidance.
Statistical studies have also provided support for the axiom of handwriting examination that no two writers share the exact same set of handwriting characteristics. See, e.g., M. Kam, J. Wetstein & R. CONN, PROFICENCY OF PROFESSIONAL DOCUMENT EXAMINERS IN WRITER INDENTIFICATION, 39 J. Forensic Scis. 5 (1994). Classification systems date back to the 1920s and continue to this day through the U.S. Secret Service Forensic Information System for Handwriting (FISH) program and similar systems overseas. With a database of large proportions, no two writers have ever been found to contain the same set of handwriting characteristics. This axiom has also been tested by computer testing, profession-wide experience, and statistical frequency-of-occurrence studies. The foregoing list is just a highlight of the numerous research studies that have tested the axiom of uniqueness in handwriting.
Some critics are claiming that one cannot establish uniqueness without testing the entire population. However, this claim clashes with standard and acceptable statistical analyses that state that population sampling can provide accurate estimates of uniqueness. While one should shy away from making any statement of absolute certainty due to population sampling and profession error rates, statistical studies do support the axiom of uniqueness if properly described in testimony.
Whether Discipline Has Been Subjected to Peer Review and Publication
Forensic document examination has a rich history of peer-reviewed publications from books dating back to the late 1800s to a regularly published, peer-reviewed scientific journal dedicated to the profession, the Journal of the American Society of Questioned Document Examiners. However, in Daubert, the U.S. Supreme Court describes peer review more broadly as “submission to the scrutiny [which] increases the likelihood that substantive flaws in methodology will be detected.” Forensic document examination as a profession submitted to this kind of scrutiny by aligning with Scientific Working Groups (SWG), a series of committees dedicated to codifying specific forms of examination and conclusion terminology. The product of the SWG was publicly distributed by ASTM International for critique and comment throughout the scientific community.
Recently, the National Institute of Standards and Technology, under the U.S. Department of Commerce and in conjunction with the U.S. Department of Justice, formed the Organization of Scientific Area Committees as a means of unifying the SWG processes under one umbrella. Forensic document examination was one of the charter disciplines included in this process and to this date is an active participant in the efforts.
The American Academy of Forensic Sciences is the most recognized forensic science organization in the United States. The Academy has been in existence since 1948 and has over 7,000 members. The Questioned Documents Section was one of the charter members of the Academy and is still an active participant.
Forensic document examination, as a profession, has a strong history of openly publishing their standard methodologies for scientific scrutiny. They have also been an integral part of forensic science organizations throughout recent history
The Known or Potential Error Rate
In the process of gatekeeping, judges must take two factors into consideration. The first consideration is the methodology error rate. The aforementioned studies by Kam and colleagues found an error rate of 6.5 percent for signature comparisons. This error rate must be considered in its proper context. First, it is the error rate for the specific question and could vary from question to question. However, Kam has conducted numerous studies and continued to arrive at approximately the same rate. In addition, one must apply the context that these problems were conducted in a conference setting and not the examiner’s laboratory surroundings or with the examiner’s own equipment. As such, these error rates should be viewed as the extreme high end or worst-case scenario. The comparable error rate of the control group was 38 percent. Found et al realized a professional error rate of under four percent.
The second consideration judges need to consider is the individual error rate. The most effective way to consider this guideline is to scrutinize the proffered expert’s training. An appropriately trained forensic document examiner will have completed a full-time, in-residence training program lasting a minimum of 24 months per the profession-published standard for training. Judges need to be very vigilant of this issue. There are large numbers of practitioners who do not meet the training standard. Red flags include distant learning, graphology (personality assessment), evening phone conferences, periodic meetings with trainer, and online courses.
The American Board of Forensic Document Examiners, established with a federal grant in 1977, is a certification board recognized by the broader forensic science community, law enforcement, and courts for maintaining principles and training requirements concurrent with the published training standards. Be wary of other certifying bodies.
The Existence and Maintenance of Standards Controlling the Technique’s Operation
Over 100 years ago, books from William Hagan, Daniel Ames, and Albert Osborn initially codified the methodologies used in handwriting comparisons to this day. The establishment of the American Society of Questioned Document Examiners, founded by Osborn, and the American Academy of Forensic Sciences in the 1940s provided the profession with two outlets from which developing technologies and examination techniques could be proposed, attempted, tested, and accepted within an academic atmosphere. Fifty years later the profession entered into the SWG within the Department of Justice as a way of specifically codifying each methodology. One specific product was ASTM Standard E-2290, Standard Guide for Examination of Handwritten Items, which provides an outline of the specific methodology used for handwriting comparisons. These standards are reviewed every five years or sooner, with a period for public comment and proposals for change.
In 2014, the National Institute of Standards and Technology initiated the Organization of Scientific Area Committees under the Department of Commerce with additional support from the Department of Justice from which all forensic science standards are to be reviewed. Forensic document examination is a charter section of this committee under the Physics and Pattern Recognition Committee. The Department of Commerce selected applicants from forensic scientists, academics, statisticians, prosecutors, criminal defense attorneys, and judges for each committee.
Standard methodologies have been central to forensic document examination for over 100 years but continue to be regularly reviewed and scrutinized by technical, legal, and academic professionals.
Degree to Which the Method Is Generally Accepted Within the Relevant Scientific Community
It would be difficult, if not impossible, to find a method of personal identification more uniformly accepted than a signature. Signatures are affixed to personal checks, receipts, prescriptions, court orders, contracts, wills, new laws (“sign into law”), driver’s licenses, and package receipts, just to name a few. We sign our name on a daily basis. Signatures are the most common form of general identification in the United States.
Forensic document examiners are used by law enforcement, criminal defense attorneys, civil attorneys, corporations, insurance companies, banks, schools, doctors, security firms, and professional disciplinary boards, to name a few. Forensic document examination is accepted within the forensic science community. Since 1948, forensic document examination has been a charter section and continual member of the American Academy of Forensic Sciences. Forensic document examination was a discipline within the SWG and is one of the subcommittees within the Organization of Scientific Area Committees. Finally, forensic document examination is a section of most federal and state crime laboratories across the country.
Misinformation is circulating that forensic document examination is not accepted without controversy. However, the facts speak differently. The majority of forensic document examiners in court are accepted with little to no opposition. There have been 24 federal courts of appeal decisions concerning the reliability of handwriting comparisons, and these courts have ruled in favor of acceptance and reliability each and every time.
In summary, whether considering the relevant community to be the general population, the legal community, case law, or corporate America, handwriting and signatures are a common and accepted process of personal identification and reliable detection of authentication.
Gatekeeping Tips from a Practitioner
While the field of forensic document examination has a solid foundation, some practitioners have limited training or fail to recognize or use the published standards. There are more practitioners with limited qualifications than true professionals, and effective gatekeeping is needed. While judges are responsible for being court gatekeepers, I, as a practicing forensic document examiner, would like to respectfully suggest ways to differentiate between the true professional and the lesser-qualified practitioners.
1. What to look for:
a. Certified by the American Board of Forensic Document Examiners
b. Member, American Society of Questioned Document Examiners
c. Member of Questioned Documents Section of American Academy of Forensic Sciences
d. Member of Southeastern or Southwestern Association of Forensic Document Examiners
e. Full-time, 24-month, on-site training
f. Trainer qualifications are available
g. Actual research work
2. What to look out for:
a. Not certified
b. Providing misleading or incomplete information as to professional organization memberships
c. Practice or training in graphology (personality assessment)
d. Statements that membership in organizations is limited to government examiners and former government examiners (no such requirements for organizations exist)
e. Distance learning/correspondence courses
f. Online courses
g. “University” in name only
h. Physically meets with a trainer periodically, not every day full time
i. Member of organizations that do not have any government examiners
Criticism of Forensic Document Examination in Court
The National Research Council published a report in 2009 establishing that forensic science was in a general need of additional scientific underpinnings and specifically stated that the scientific basis for handwriting comparisons needs to be strengthened. Research by Srihari and Johnson, myself, and our colleagues has strengthened the foundations of the science. However, this focus on limitations has had an unfortunate side effect.
In recent years, a new and different critic of forensic science has appeared in the courtroom. This person is the self-proclaimed “expert critic” claiming a level of expertise in the deficiencies of the forensic science principles that qualifies him or her to testify as an expert witness. A search for organizations of expert critics turns up nothing, as do searches for “expert critic” degrees, courses, certifying bodies, legal subsections of bar associations, journals, or other mainstays of the legal/scientific professional world. The author wishes once again to respectfully suggest ways of differentiating between legitimate criticism and junk criticism.
1. What to look for:
a. The critic has formal training in forensic document examination.
b. The critic has training in statistics if providing statistical information.
c. The critic documents proper standards of procedure for processes.
d. The critic provides documentation in support of training.
e. The source of documentation is peer-reviewed scientific journals.
2. What to look out for:
a. The witness claims to or has claimed to be an “expert critic.”
b. The witness claims expertise in some form of criticism or authority in weaknesses of sciences.
c. The witness cannot provide documented training.
d. The data are unsupported.
e. The source of data is law journals or other unscientific and non-peer-reviewed sources.
Conclusion
The purpose of expert testimony is to assist the trier of fact by explaining the evidence. A properly trained forensic document examiner has the training and methodology to provide reliable information to the trier of fact. Judges have the duty to filter out misinformation or partial truths laced into the mix by unqualified and underqualified “experts” or “critics.” Qualified forensic document examiners stand prepared to team with the judiciary in a cooperative undertaking to help ensure scientific evidence is more reliable and effective.