When Is the Right Time to Disclose Using TAR?
While litigants have grown increasingly comfortable with predictive coding, questions remain about when the TAR process can be introduced and whether litigants will encounter resistance from either opposing counsel or the court. A common scenario involves parties who decide to use TAR after the Rule 26(f) conference and after stipulations to discovery methods have already been made. The Georgetown panel tackled these issues in a similar scenario where, after the parties stipulated to use agreed-upon search terms, the producing party asked for an extension and to use predictive coding to manage the burdensome review set. The panel and Judge Francis weighed in on whether courts would grant a process change after a search-term stipulation is in place and, if so, whether parties are obligated to also stipulate about specifics of the predictive-coding process.
Judge Francis instructed that parties need to heed caution when asking for process changes that would result in a do-over and that the court’s gut tendency is to hold parties to an existing stipulation in the interest of cost containment and proportionality. Judge Francis highlighted that the act of changing course may often incur extra cost, especially if both parties will need to engage in further discussions or vet a new process. That said, parties should also be warned that the court is not in the best position to judge the validity of TAR tools or processes, and if the parties cannot agree to a process, the producing party will have the option to use what it considers most valuable.
Judge Francis’s comments about the court’s reluctance to resolve this kind of dispute is reminiscent of United States v. ExxonMobil Pipeline Co., No. 4:13-CV-00355, 2014 WL 2593781 (E.D. Ark. June 10, 2014). In ExxonMobil, the defendants sought an extension of the deadline to produce documents and a potential extension of the entire case schedule, due to the burden of reviewing a large volume of documents. The defendants informed the court that they had proposed to the plaintiff using predictive coding, but the plaintiff had not yet agreed. The court declined to rule on using predictive coding, however, because there was “no indication in the parties’ submissions that they seek the Court to resolve disputes as to the method of review to be utilized[.]”
In the mock scenario, Judge Francis granted the extension and the use of predictive coding, in line with the decision in Bridgestone Americas, Inc. v. International Business Machines Corp., No. 13-1196, 2014 WL 4923014, at *1 (M.D. Tenn. July 22, 2014), where the court, relying on Federal Rule of Civil Procedure 26, stated “discovery [should] be tailored by the court to be as efficient and cost-effective as possible.” On that basis, the court concluded that the plaintiff could use predictive coding to search the documents, even though keyword searches had already been performed.
The panel emphasized that litigators should gather as much information as possible prior to a Rule 26(f) conference, including working with experts and vendors to establish a proper workflow prior to the conference, to avoid mid-discovery methodology changes. Because effective protocols can only be developed with proper guidance and understanding of the technologies being offered, parties who are educated and supported by expert advice are in better positions to establish a defensible process for search and retrieval. Both Progressive Casualty Insurance Co. v. Delaney, and ExxonMobil reflect the importance of developing a workable search and review methodology early rather than late in the discovery process. In Progressive, Magistrate Judge Peggy Leen described the potential effectiveness of predictive coding positively and stated that she would not have hesitated to approve a predictive-coding protocol had it been submitted earlier in the case. See Progressive Cas. Ins. Co. v. Delaney, No. 2:11-cv-00678, 2014 WL 3563467 (D. Nev. July 18, 2014).
The panelists also advised that confining Rule 26(f) conference discussions to addressing the scope of discovery rather than to the specific processes could help practitioners avoid the issue of justifying mid-discovery methodology changes.
How Much Disclosure about the TAR Process Is Required?
The extent and requirement of transparency concerning training sets and nonresponsive ESI remain openly debated issues this year and undoubtedly add to the hesitation that litigants have about using predictive coding. Despite last year’s decision in Biomet, where parties were not required, under the Federal Rules of Civil Procedure, to share predictive-coding training documents with opposing counsel because “it is beyond the scope of any permissible discovery,” the misconception that predictive coding is more burdensome persists. In re Biomet M2a Magnum Hip Implant Prods. Liab. Litig., No. 3:12-md-2391, 2013 WL 6405156, at *1 (N.D. Ind. Aug. 21, 2013). Unfortunately, the court in Progressive perpetuated this misconception by ignoring Biomet and stating, “the courts have required the producing party to provide the requesting party with full disclosure about . . . the documents used to ‘train’ the computer.” See Progressive, 2014 WL 3563467, at *10 (citing DaSilva Moore v. Publicis Groupe SA, No.11-cv-1279, 2012 WL 1446534 (S.D.N.Y. Apr. 26, 2012),and In re Actos (Pioglitazone) Prods. Liab. Litig., No. 6:11-md-2299,2014 WL 355995 (W.D. La. Jan. 30, 2014)). What the Progressive court failed to acknowledge was that in DaSilva Moore and Actos, the producing parties voluntarily shared the training sets.
Tackling a similar issue in a mock scenario, Judge Francis did not permit plaintiffs access to training sets or nonresponsive ESI, stating that they are protected by the work-product doctrine. Judge Francis rejected the plaintiffs’ argument that they were entitled to the training and nonresponsive ESI because a predictive-coding process would magnify and expand errors in the training sets, and the plaintiffs should therefore be allowed to vet the process. Judge Francis also rejected the plaintiffs’ argument that training sets and nonresponsive documents were comparable to information contained in a document-retention policy and therefore also discoverable under the Federal Rules.
The judge went further to say that training sets not only go to whether a document is responsive and discoverable but may also give insight to how counsel prioritized documents for review and potentially also represent documents that counsel deemed central to the litigation, all of which encroach on work product. Further, Judge Francis noted that courts would also be unlikely to mandate disclosure of any statistics of predictive coding, such as recall, absent a showing that a production was inadequate. The judge emphasized that he expected that the parties will have experts testify to the statistics and the validity of the process and prefers that the court not be involved in the “weeds of the process.”
The panel also discussed other valid reasons why sharing nonresponsive ESI may be a dangerous practice, including that ESI not responsive to the matter at hand may be relevant in other matters in the future. The panel warned that it is not unusual for counsel to take discovery in one case in hopes of learning something about another case, a concern that could be especially relevant in unfair competition cases.
Will Judges Support the Use of Predictive Coding?
Litigants continue to grow increasingly more comfortable with predictive coding, and questions surrounding predictive coding have recently moved away from whether the technology is trustworthy, to trying to define the benchmarks that make a predictive-coding project successful. Those hoping for insight from a judicial perspective on this issue were disappointed when Judge Francis stated that judges often don’t know what standards for recall and precision to accept, and that judges “usually pull those numbers out of a hat.” The panel members suggested that a successful recall standard can only be decided at the end of a predictive-coding project, after evaluating the nonresponsive set, and warned that these metrics should not be analyzed in a vacuum. Judge Francis further noted that courts may be in the process of considering appointing special masters to ensure fairness as technology grows increasingly complex.
Dispelling any residual notion that judges may not view predictive coding favorably, Judge Peck, from the audience, affirmed that “in every case, where the responding party wanted to use TAR, and the issue was raised with the court, it was allowed.” Judge Peck acknowledged that while issues of transparency still exist, it is clear from the research that “TAR is, at the moment, significantly better than manual review or keywords.” In support of his landmark adoption of predictive coding in DaSilva Moore, Judge Peck recited Dynamo Holdings, “where the judge with argument from the IRS that predictive coding is ‘unproved technology,’ allowed predictive coding to be used and responded, ‘we understand that the technology industry now considers predictive coding to be widely accepted for limiting e-discovery to relevant documents and effecting discovery of ESI without an undue burden.’” Dynamo Holdings Ltd. P’ship v. Comm’r of Internal Revenue, Nos. 2685-11, 8393-12, at 10 (T.C. Sept. 17, 2014).
The Georgetown panel and case law provides great insight into how far TAR has come. As these analytic processes continue to develop and gain traction in 2015, we will likely see courts develop a more sophisticated understanding and treatment of the topic.
Keywords: litigation, pretrial practice, discovery, technology assisted review, TAR, e-discovery, ESI
Jenny Le is discovery counsel at Evolve Discovery, based in New York, New York.