Surveys have changed the landscape of valuable data held by consumer-facing businesses. Both franchisors and franchisees are using surveys more and more, as they have become increasingly easy to access and increasingly sound in their methodology. Franchisors retain professional survey companies to independently assess the experiences of their franchisees’ customers, the efficacy of their own marketing, and the quality of their system-wide product offerings. Franchisees conduct their own surveys at the cash register, through e-mails, or through online services such as SurveyMonkey to assess customer service, store cleanliness, and the effectiveness of their local marketing efforts. As a result of these and many other surveys, millions of data points are collected daily around the world.
From a business perspective, the practical benefits of consumer surveys are easy to see. But where does all of this valuable data go when litigation unfolds? Most often, it sits on the virtual shelf gathering micro-dust. When this happens, a valuable resource is lost. Consumer surveys, when properly conducted, can be effective weapons in litigation, for franchisors and franchisees alike. Courts have shown a willingness to allow triers of fact to hear survey results and then determine the weight they should be afforded based on the methodology they use.
The Admissibility of Surveys
Practitioners are likely most familiar with the use of consumer surveys in trademark cases, to prove or disprove a likelihood of confusion between rival marks. But consumer surveys were not always considered trustworthy or admissible. Courts in the first half of the twentieth century treated them as inadmissible hearsay. See, e.g., Dupont Cellophane Co. v. Waxed Products Co., 6 F. Supp. 859, 884 (E.D.N.Y. 1934), modified, 85 F.2d 75 (2d Cir. 1936) (refusing to admit survey because court could not “see how plaintiff could even test the facts, as it had no opportunity for cross-examination of those who were supposed to have answered the questions”); Elgin National Watch Co. v. Elgin Clock Co., 26 F.2d 376, 376-77 (D. Del. 1928) (refusing to admit survey because survey constituted hearsay). By the early 1950’s, however, the tide began to turn. Since the seminal decision in Zippo Manufacturing Co. v. Rogers Imports, Inc., 216 F. Supp. 670 (S.D.N.Y. 1963), the trend has been to admit properly conducted consumer survey evidence.
In 1999, now United States Supreme Court Justice Sotomayor authored one of the leading federal court opinions on the modern approach to the admissibility of surveys, Schering Corp. v. Pfizer Inc., 189 F.3d 218 (2nd Cir. 1999). In Schering, the trial court denied Schering’s motion for a preliminary injunction against Pfizer and UCB Pharma. In support of its motion, Schering offered the results of five surveys in which physicians were asked to provide their recollections, and in some cases their impressions, of communications that the defendants’ sales representatives made to physicians during sales calls. Justice Sotomayor analyzed two of the most often-cited grounds on which such evidence is now admitted: the state of mind exception to hearsay under Federal Rule of Evidence (“F.R.E.”) 803(3) and the residual hearsay rule, F.R.E. 807.
F.R.E. 803(3) excepts any “statement of the declarant’s then-existing state of mind (such as motive, intent, or plan) or emotional, sensory, or physical condition (such as mental feeling, pain, or bodily health), but not including a statement of memory or belief to prove the fact remembered or believed unless it relates to the validity or terms of the declarant’s will.” Schering, 189 F.3d at 227. As Justice Sotomayor explained, surveys, if properly conducted, “poll individuals about their presently-existing states of mind to establish facts about the group’s mental impressions,” and should therefore be admitted. Id. Accepting surveys as evidence under this hearsay exception, “obviates the need to examine [survey] methodology before overruling a hearsay objection.” Id. at 227-28. Instead, “errors in methodology . . . go only to the weight of the evidence.” Id. at 228 (citing Grotrian, Helfferich, Schulz, Th. Steinweg Nachf. v. Steinway & Sons, 523 F.2d 1331, 1341 (2d Cir. 1975)).
If survey results are also offered to show actual facts recalled or believed (as opposed to the mere existence of a statement of mind), the evidence would not be admissible under F.R.E. 803(3). Thus, the Schering opinion next analyzes the residual hearsay rule, F.R.E. 807, as a second basis on which such evidence may be admitted. Id. at 230-31. This rule allows the introduction of an otherwise inadmissible hearsay statement if the statement satisfies the following five elements: trustworthiness; materiality; probativeness; importance; and the interests of justice and notice. Id. at 231.
Justice Sotomayor’s opinion emphasizes certain survey methodologies that can be used to increase the “trustworthiness” factor of a survey, which case law reflects is one of the more scrutinized of the five requirements. See, e.g., Baumholser v. Amax Coal Co., 630 F.2d 550, 552 (7th Cir. 1980) (“To qualify a study or opinion poll for admission into evidence, there must be a substantial showing of reliability.”); Zippo, 216 F. Supp. at 684. First, insincerity or “untrustworthiness” can be reduced if the interviewees lack knowledge of the subject litigation. Schering, 189 F.3d at 233. Second, clear and non-leading survey questions further reduce the risk of “faulty narration.” Id. The trustworthiness element is “for the most part satisfied if the [survey] is conducted in accordance with generally accepted survey principles, and if the results are used in a statistically correct way, since proper survey and statistical methods are intended to assure a poll’s reliability.” Id. at 235 (quoting Pittsburg Press Club v. United States, 579 F.2d 751, 758 (3d Cir. 1978)).
Surveys in Franchise Litigation
Vehicle dealers, whose relationships with manufacturers and distributors are often governed by state and federal relationship statutes similar to franchise relationship statutes, have successfully used survey evidence to prove the unlawful conduct of their manufacturer or distributor. In Randy’s Studebaker Sales, Inc. v. Nissan Motor Corp., 533 F.2d 510 (10th Cir. 1976), for example, a Nissan franchisee, Randy’s Studebaker Sales, Inc. (“Randy’s”), alleged violations of the Sherman Antitrust Act and federal Automobile Franchise Act, 15 U.S.C. §§ 1221-25, in connection with Nissan’s refusal to renew Randy’s automobile franchise unless Randy’s acquired larger facilities, hired additional personnel, and increased working capital and sales performance.
To help prove that Nissan’s conditioned renewal was unlawful under the Automobile Franchise Act, Randy’s introduced a questionnaire, prepared after litigation had begun, in which select customers attested to the quality service that Randy’s provided. In appealing the damages awarded to Randy’s in the trial court under the Automobile Dealers Franchise Act, Nissan challenged the court’s admission of the questionnaire. The appellate court determined that “the questionnaires were properly admitted [under F.R.E. 803(3)] to reflect the then-existing state of mind of the customers.” Id. at 520. Although Randy’s did not poll all of its customers, “the surveys of customers during a particular period of time without restriction as to the group was used and so there is little basis for objection.” Id. at 521. In any event, the court noted, “technical inadequacy” bears on the weight of the evidence, not its admissibility. Id.
Auto manufacturers and distributors, akin to franchisors, also have used surveys to prove the lawfulness of their own conduct. In Tappan Motors, Inc. v. Volvo of America Corp., 74 A.D.2d 846, 425 N.Y.S.2d 970 (N.Y. App. Div. 1980), a Volvo distributor successfully used surveys showing customer dissatisfaction to obtain a preliminary injunction preventing a dealer from continuing its automobile franchise prior to trial. To prove its likelihood of success in showing that it terminated the dealer for cause in accordance with New York’s motor vehicle franchise statute, the distributor used a survey of purchasers showing dissatisfaction with the dealer and its customer service. See Tappan Motors, Inc. v. Volvo of America Corp., 102 Misc.2d 570, 573, 423 N.Y.S.2d 819, 821 (Sup. Ct. 1980) (denying application for preliminary injunction), rev’d, 74 A.D.2d 846, 425 N.Y.S.2d 970 (N.Y. App. Div.); see also Ed Koehn Nissan, Inc. v Nissan Motor Corp., 1994 U.S. Dist LEXIS 11575 (W.D. Mich. June 22, 1994) (manufacturer’s refusal to allow dealer to relocate its dealership based, in part, on market survey showing that the proposed market area was not desirable was deemed reasonable and not in violation of the Michigan Motor Vehicle Act).
At least one court has even endorsed the use of consumer satisfaction surveys by manufacturers in determining a dealer franchisee’s eligibility for performance-based incentives. In Colonial Imports Corp. v. Volvo Cars of North America, Inc., 2001 WL 274808 (D.N.H. Jan. 9, 2001), a dealer argued that Volvo’s decision to tie dealer eligibility for cash incentives to the results of subjective consumer surveys was arbitrary and, thus, unlawful under New Hampshire’s Vehicle Franchise Act, which makes it unlawful for motor vehicle manufacturers and dealers to “engage in any action which is arbitrary, in bad faith, or unconscionable and which causes damage.” N.H. Rev. Stat. Ann. § 357-C:3, I (2000). Specifically, the dealer argued that because the surveys were subjective and manipulable, their use was arbitrary. Colonial Imports Corp., 2001 WL 274808, at *6. But the court rejected the dealer’s argument, holding that “subjective” does not necessarily lead to arbitrary. “Given the inherently subjective nature of customer satisfaction and its importance to any commercial endeavor, it was not unreasonable for Volvo to base dealer incentives on the results of surveys that attempted to measure customer satisfaction.” Id. at *6.
Now that courts are willing to allow the introduction of consumer survey results into evidence, what steps can franchisors and franchisees take to ensure that their survey results are both admissible and afforded the greatest evidentiary weight?
At the outset, franchisors and franchisees not already doing so should consider implementing consumer satisfaction surveys as part of their regular business practice – not only to benefit their business operations, but to provide valuable evidence if litigation ever arises. Second, when preparing their surveys, franchisors and franchisees should consider consulting counsel and survey providers familiar with the issue of survey admissibility. At a minimum, franchisors and franchisees should consider using quick and inexpensive (or sometimes free) survey resources available to the general public, such as SurveyMonkey, Google® Consumer Surveys and UsampTM to name a few, while keeping in mind that careful survey preparation and methodology are critical.
In preparing and implementing surveys, franchisors and franchisees should keep the following in mind:
- Don’t wait until litigation arises. As Justice Sotomayor noted, surveys of interviewees that lack knowledge of the litigation or “purpose” of the survey results are inherently less biased.
- Frame survey questions to inquire about the interviewee’s present impressions, rather than recollections of past events. This is easier to accomplish if the surveys are conducted as part of a company’s day-to-day operations, rather than as part of litigation.
- Frame survey questions that are “clear, precise, and non-leading.”
- If there is risk that consumer-interviewees will lie for personal reasons, consider making the survey anonymous to reduce that risk.
- Make the sample size statistically significant.
- Do not limit the survey arbitrarily. Limitations as to a certain geographic area or specific period of time, for example, may be appropriate restrictions.