Testing Your Evidence: Before and Even During Trial

Mock Trials

The most popular and effective technique for testing your evidence is the mock trial. If you use a consultant, have him prepare a jury questionnaire similar to that used by some federal courts to assist in recruitment of the mock jurors. There should be questions regarding prior jury service and prior involvement in legal proceedings. The questionnaire should also cover demographics—e.g., age, marital status, race, educational background, income, and employment history—together with questions tailored to the specific case. In an accounting malpractice case, for example, there should be questions focusing on experience with billing, accounting, finance, bookkeeping, and financial statements.

Almost all cases involve expert testimony. A properly framed questionnaire, therefore, should probe mock jurors’ level of understanding of disciplines that could be the subject of expert testimony, such as medicine, engineering, and accounting. Thus, in a personal injury case, mock jurors should be asked questions such as the following: “Have you heard the term diagnosis? What do you think it means? Do you know the difference between diagnosis and prognosis?” In a product liability case involving a fire, mock jurors should be asked similarly tailored questions: “Have you heard the term ‘point of origin’? What is your understanding of the phrase?”

No one mock trial fits all cases. The professional consultants will likely recommend that the mock trial be “balanced,” and in general they’re right. In other words, each portion of the mock trial should be like the real thing, where both sides present opening statements, evidence, and closing arguments. Live testimony is preferable because it is most like a real trial, but key evidence from depositions—good and bad—should also be used.

Occasionally, there may be an overriding consideration arguing against presenting a balanced case to the mock jurors. If you have experts with limited experience testifying at trial, for instance, a mock trial can test how they perform. If they don’t hold up, more preparation will be necessary.

If your mock trial lasts more than one day, the second can be used to restore balance by presenting testimony from persons acting as your opponent’s experts. The mock jurors should be questioned about their understanding of the experts’ opinions and, in certain types of cases, the exhibits used. (More about exhibits later.)

Fact witnesses are important too. To maintain balance, at least one significant fact witness from each side should be presented, and the mock jury should answer questions about their reactions to the testimony. “Did you find it believable? Did it convince you? Was some of it irrelevant? Was it understandable?” The mock jury should also rate the witnesses on credibility, likability, truthfulness, knowledge, ease of understanding, evasiveness, and demeanor.

The lawyers who conduct the mock trial should be the same who are going to try the real case. If you are the sole, or lead, trial attorney for your client, strongly consider playing your opponent. I’ve always believed the best advocates must be able to appreciate their opponent’s position. What better way to do that than by preparing her case for trial?

Deciding on which evidence to “test on the mock jury” forces early consideration of what additional evidence may be necessary. In an employment discrimination case, for example, the performance evaluations of the plaintiff are almost always going to be offered into evidence at trial. They must be used in the mock trial. Other portions of the personnel file, such as disciplinary records, should also be showcased. The employee plaintiff will have her interpretation of these key documents; company counsel must identify the best live witness to present the employer perspective on the same exhibits to a mock jury. In personal injury actions, aerial photos or drawings of the accident site will be used at trial—use them in the mock trial.

By the time you conduct the mock trial, it is likely that you are aware of at least one witness whose trial testimony may deviate from his deposition testimony. Assume that will occur, impeach the witness at the mock trial, and ask the mock jury to comment specifically on the impeachment. Because impeachment by prior deposition testimony sometimes falls flat, you may need to impeach more than one witness or use more than one mock jury to obtain information that is at least somewhat reliable.

Your client or client representatives will likely want to attend the mock trial. After all, they’re paying for it. Don’t despair—seize the opportunity to obtain feedback from your client in a setting far different from a conference room. This is particularly helpful in complex cases where jurors need to be educated on basic terms and definitions from specialized fields. Clients can be critical, and some of it may not be constructive. Experienced trial lawyers know a thick skin is required, so make sure you wear one to the mock trial.

Constructive criticism of your performance and the evidence should also be solicited from your jury consultant. An assessment of the proof is typically not hard to obtain, but criticism of the lawyer’s work seems to be rare, perhaps because consultants hope the customer will return to the store. If you want this type of feedback, however, insist on it prior to signing your agreement with the consulting firm. It should be received in person by the consultant, though sometimes travel schedules relegate this to a phone conference. In addition, get the feedback in writing. You’ll want to review the consultant’s critique with a cool head when the immediate sting has worn off.

Evaluating your performance isn’t the primary goal of a mock trial, though. The main purpose is to test your evidence, themes, and theories. This doesn’t mean presenting evidence with no chance of being admitted at the actual trial; it means you present the evidence as you would during the real thing. If you do want to gauge admissibility, find someone to act as judge. There are retired judges with the time, skill, and willingness to do this, and it will also provide another (albeit unusually educated and experienced) mock juror. The judge’s feedback can be just as valuable as what you hear from the other mock jurors.

We’re all accustomed to arguing in the alternative and advancing numerous claims and defenses. As the litigation progresses, this approach becomes less and less advisable. The filing by defendants of the now almost inevitable summary judgment motion requires paring down the evidence. When the summary judgment motion fails, the evidence must be reexamined. Use the mock trial to do so. Remember, it is an exercise to test your evidence, themes, and theories. If the mock jurors soundly reject two of your claims or defenses based on the evidence they heard, it is probably a good idea not to present them at the actual trial.

At the risk of stating the obvious, feedback from the mock jurors is essential. Feedback does not mean that the lawyers engage the mock jurors in an informal question-and-answer session, as is often done after an actual trial. Feedback means that the jury consultant should prepare detailed written questions and obtain written answers, and then use them for her own question-and-answer session with the mock jurors. This session should be observed by the lawyers, their paralegals, the client representatives, and anyone else who has been invited to the party behind the glass.

Once all of the feedback has been obtained, the jury consultant should meet with the attorneys and their clients to summarize the responses, quantitatively and qualitatively, together with the results of her questioning during the mock trial itself. Some jury consultants to keep “a running score”—in other words, which side does the mock jury favor after opening statement, after the plaintiff’s case, after the defendant’s case, and so on. Whose evidence was more persuasive? Why? Whose witnesses were more believable? There should be more than one question per subject because the mock jurors will likely answer the first with general impressions, and you want more details.

Pay attention to the mock juror comments and questions regarding the evidence presented. Perhaps even more so, pay attention to the questions and the comments from the mock jurors about evidence not presented. Jurors, both mock and actual, are sometimes suspicious of information “hidden” by trial attorneys.

One last point—most cases involve some evidence that could anger or offend a jury. Use the mock trial to test the jurors’ reactions, but remember that the typical mock trial is only one day, which is not likely to be sufficient time to test whether potentially offensive evidence can be blunted by an additional week of testimony and exhibits, assuming a lengthy trial.

Summary Jury Trials

Summary jury trials, sometimes also referred to as a mock jury exercise, are another technique used to test the persuasiveness of evidence. The lawyers present statements that are a combination of opening statement and closing argument. While no witnesses testify, exhibits are presented and taken into the jury room during deliberations. Jury instructions are read, and the summary jury deliberates and reaches a verdict. For certain cases, two summary juries can be used to provide feedback from more than one group.

The summary jury trial is now typically done in a setting similar to that of the mock jury trial with the use of a professional consultant who, if an experienced attorney or retired judge is not available, can read the jury instructions. Moreover, the consultant can and should prepare the same type of written questions about the evidence, as in a mock trial.

The number of exhibits, as compared with witness testimony, is sometimes greater than in a full mock jury trial. Summary jury trials, therefore, are very useful for focusing on documentary evidence, such as performance evaluations in an employment case, medical records in a personal injury case, or engagement letters in legal and accounting malpractice cases.

The jurors should be asked specific questions about the exhibits: “Did you think that the evaluations were intentionally slanted? Did you think that the file was ‘papered’ ?Did you think that the medical records were altered? Were the engagement letters full of jargon and, therefore, difficult to understand? If so, would you have preferred exhibits that contained only excerpts of the letters?”

Because it is truly a summary proceeding, there can be a tendency to omit background information about a witness or client. Years ago, we used a summary jury trial in a trademark infringement case. We represented a local, but large, health insurance company against a substantially larger out-of-state competitor. After the verdict, one of the summary trial jurors mentioned that our client’s size was a disadvantage. We responded that the defendant was much larger, to which the summary trial juror stated, “Why weren’t we told that?”

The summary jury trial is especially useful in cases where funds are limited. In commercial litigation where the clients are large but budget-conscious companies, cost can be reduced by using non-lawyer employees who, with careful selection by counsel, can be assembled to resemble the composition of the actual jury. See Jeh Charles Johnson, Mock Juries: Why Use Them?, 35 Litig (Winter 2009), at 32, 33. The combination opening statement/closing argument, together with exhibits, is presented in the same manner described above. The jury instructions can be read by an attorney not involved in the case who can also receive the verdict and preside over the question-and-answer session afterward.

Shadow Juries

The shadow jury offers another tool to analyze your evidence. Shadow jurors observe the actual trial and provide feedback, through an intermediary, to the lawyers and their clients during breaks and evenings. The shadow jurors should be told that they are participating in a jury research project. They must be told to act like the real jurors, taking their breaks when the real jury does and otherwise heeding the judge’s instructions on outside information. With modern technology, the shadow jurors can observe the trial by reviewing video recordings or even a simulcast. See Dahlia S. Fetouh & Christopher Land, Mock Jury Exercises, practicallaw.com (Feb./Mar. 2014), www.whoi.edu/fileserver.do?id=197864&pt=2&p=205930. In my view, however, the most effective type of observation is to have the jurors in the courtroom. While this can lead to questions from opposing counsel and even the judge, there are sufficient answers.

For example, opposing counsel can simply be informed that “these persons in the back” are observing the trial. Indeed, the work-product privilege protects against more invasive questioning and any obligation to respond to such questions. See, e.g., Burk v. C.B. Fleet Co., No. 607CV00611, 2009 WL 1109001 (S.D. W. Va. Jan. 12, 2009) (granting motion in limine barring any mention of the shadow jury).

As far as the trial judge is concerned, use of the shadow jury can be cleared before trial. This is usually unnecessary, given that the overwhelming majority of trials are open to the public. But because the trial judge controls the courtroom, clearance may nevertheless be advisable.

The shadow jurors must be selected, and compensated, by an intermediary. This is most often done by a jury consultant, but it need not be. Others, such as a marketing consultant working with the lawyers, can assemble a shadow jury demographically similar to the actual one and prepare questions for them. But it is crucial shadow jurors not be told which side has hired them.

The principal advantage to the shadow jury is its ability to see the evidence as it is presented to the actual jurors. Unlike real jurors, the shadow jurors can react to, and comment on, the actual testimony and exhibits in real time. They can take notes about matters to discuss with the intermediary and answer questions about the evidence. The feedback provides an opportunity to adjust lines of testimony and, especially with today’s technology, modify the use of exhibits.

Early in my career, I was co-counsel for a plaintiff in a serious injury case stemming from a car accident allegedly caused by the improper design of a roadway. One of our experts, an accident reconstructionist with excellent academic credentials and more than 30 years’ experience as a professional engineer and trial witness, gave confusing testimony, sounding more akin to a tedious physics teacher than a seasoned and successful expert. This despite what we thought was thorough preparation. The expert’s testimony was important to the issue of proximate cause, so imagine our disappointment when one of the shadow jurors told our intermediary that “either side could have hired that guy.”

But now that we knew the extent of the problem, we could devise a solution. We modified the testimony of our other expert, a traffic engineer, to clear up the confusion and buttress the earlier expert’s opinion on proximate cause. In the same case, one of the shadow jurors told the intermediary that the roadway in question could have been redesigned at a relatively low cost, thereby validating our expert testimony and even anticipating part of our closing argument.

The principal disadvantage of shadow juries is the limited time to consider modifications to the evidence to be presented at trial. After all, you spent years in litigation and many, many hours in trial preparation. Perhaps, as educators and psychologists will tell you, your first impression was the right one. It is not, however, your first impression that is being tested. It is that of the shadow jurors.

Another disadvantage, albeit slight in my view, is having the lawyers’ attention diverted from the actual task at hand—winning the case before the actual jury. The answer to this—as with many other things in the life of trial lawyers—is simply concentration and a redoubled focus on the important work at hand.

Focus Groups

Finally, consider trying a fourth device for testing evidence—the focus group. These have been used for many years by marketing firms to test consumer attitudes about planned and existing products and services. In this context, the marketing firm typically assembles people in a conference room, asks them numerous questions, compiles the responses, and uses basic statistical techniques to measure consumers’ attitudes.

Jury consulting firms can and do easily adopt this model. Typically, the consultant will explain the facts of the case to the assembled group and conduct a question-and-answer session. This can be done face-to-face with the use of written questionnaires and answers. If done this way, especially when used to evaluate the persuasiveness of evidence, counsel should offer input into the wording of the questions. If that isn’t possible, insist on having more than one question per evidentiary subject matter because the individuals in the focus group will tend to give general impressions and be difficult to pin down.

Some focus groups use handheld devices so members can provide instantaneous responses. Indeed, these can be used with any of the four techniques described in this article. See Fetouh & Land, supra, at 46. Instantaneous reactions to evidentiary matters, however, may not be desirable data points. They deprive the responder of an opportunity to think through whether certain testimony or a certain exhibit was convincing. Real juries will have time to mull things over this way. Exhibits should be read, not skimmed. A handheld device may encourage skimming, which is particularly dangerous with financial documents.

Often there are several witnesses who can provide testimony on the same or similar topic. Focus groups provide an opportunity to test similar lines of testimony through different witnesses, giving an indication as to relative persuasiveness, credibility, and comprehensibility. The same is true of exhibits in the sense that they can be presented in different packages or groupings. The focus group can react to a complete document (such as an income statement or nurse’s note), to an edited version of them, or to a summary. Editing makes the exhibit easier to understand but may make jurors suspicious. Use the focus group to probe these issues. See further, Timothy S. Tomasik, The Plaintiff’s Perspective: Jury Research Can Win Your Case, 41 The Brief (Spring 2012), at 60, 61-2.

Because chronology is important in nearly all cases, timeline exhibits are used in virtually all trials. Focus groups offer an excellent opportunity to test differently designed timeline exhibits. Indeed, a focus group can be assembled prior to a mock trial or summary jury trial purely for the purpose of testing different versions of key exhibits. The results can then help shape which exhibits are used.

There are some obvious disadvantages to focus groups compared with the other three mechanisms. Because focus groups usually last only a half day, they tend to be limited to discrete matters. Plus, there are no juror deliberations. Attitudes and reactions are tested more on an individual than a group basis. With the help of a jury consultant, the collective aspect can be emphasized more. But at that point, a summary jury trial may make more sense because it can be completed in one day with special interrogatories to the jurors. Lastly, a true focus group usually does not allow for attorney presentations. Thus, there is no opportunity to measure how members react to the trial attorneys.

Jury consulting services are less expensive than they used to be, and using them is no longer reserved for the so-called “big” cases. Moreover, when funds are limited, the four techniques described in this article can be applied without a consultant. Regardless of the approach or even the specific technique used, it is well worth the effort. Failing to use or at least investigate some type of pretrial evidence testing does not yet violate the duty of competent representation, but stay tuned. Down the line, it may do so, especially as it becomes even more cost-effective.