Feature

Pros and Pitfalls of Artificial Intelligence in IP and the Broader Legal Profession

By Timothy J. Carroll and Manny Caixeiro

Published in Landslide Vol. 11 No. 3, ©2019 by the American Bar Association. Reproduced with permission. All rights reserved. This information or any portion thereof may not be copied or disseminated in any form or by any means or stored in an electronic database or retrieval system without the express written consent of the American Bar Association.

Artificial intelligence (AI) is, at its core, a computer that has been programmed to mimic the “natural” intelligence of human beings, such as by learning, reasoning, or making decisions. Multiple compa- nies have begun marketing AI sys- tems to attorneys as a cost-effective means to perform document review, to wade through voluminous information, to interpret contracts, and to perform legal research. Usage of AI in the legal community is forecasted to rise in the future. A recent survey by Thomson Reuters of in-house corporate counsel found that over 60 percent of respondents believe that usage of AI will be mainstream within the next 10 years, and 21 percent believe it will be within the next five years.1 As might be expected, the anticipated benefits of AI are seen as saving time, reducing costs, and mitigating risks.

In recent years, the general public has become more familiar with AI. We have seen sophisticated AI tools being used to defeat the best human players on the game show Jeopardy,2 we have witnessed the launch of self-driving taxi services, and video games that adapt to the behavior of players are now commonplace.3 The legal community has long perceived itself as a bastion of inimitable natural intelligence. However, the reality is that lawyers have utilized artificial intelligence for years. The Boolean search features and other logical search operators in LexisNexis and Westlaw are basic examples. More sophisticated AI tools are constantly being developed and have the potential to save lawyers time and money by reducing routine tasks that are repetitive and simple and also mitigating or avoiding risks. But these advantages of AI are not without risks and limitations. As discussed below, strong and efficient practitioners must learn to harness the power of AI, but must be wary of overreliance on these technologies.

Why Use AI in the Legal Context?

The potential for human error is one risk that can be addressed and mitigated by usage of AI. Human lawyers are relatively expensive, inefficient, and slow compared to AI for certain tasks, such as document review. A study conducted in 1985 asked experienced attorneys and paralegals to use keyword searching in a database of approximately 40,000 documents (roughly 350,000 pages of text) to find the documents that were responsive to 51 document requests. The results were unimpressive, as the search teams only succeeded in identifying less than 20 percent of the relevant documents (far less than their 75 percent goal).4 AI, however, is able to produce much higher results.

Repeatable Results with AI

A major strength of AI is the repeatability of results. Because AI follows strict rules (i.e., algorithms), a given AI system should provide the same results based on the same inputs. Inexperience or physical exhaustion will not affect the performance of the AI systems.

However, the results provided by AI are only as good as the underlying algorithm and data. In the past, the only way to improve the search results was for programmers to make manual changes or data entry. More modern AI systems, however, employ machine learning—in which the computer system effectively “teaches” the AI system. This, in turn, has generally improved search or query outcomes.

These improvements have helped attorneys and clients to answer the all-important question: “what is my chance of winning this lawsuit?” In the past, the odds would have been based on an attorney’s subjective estimation of the case’s merits. However, AI has been shown to predict the results of U.S. Supreme Court cases when provided with sufficient data on which to base the conclusion more accurately than human prognosticators. In 2017, three law professors provided a machine learning system with data relating to 28,000 U.S. Supreme Court decisions from 1816 until 2015.5 Based on this data, the system was able to determine the Court’s decision with 70.2 percent accuracy, outperforming the optimized null model by approximately 5 percent. In comparison, knowledgeable legal experts are only accurate around 60 percent of the time, according to a 2004 study.6 It seems to be a matter of time until case-predicting software of this nature is widely commercialized and becomes part of the tool set for every practitioner.

Efficiency with AI

AI also works much more quickly and efficiently than comparable manual human operations. AI excels at performing tasks that include clearly definable parameters, as lawyers are well aware from using Lexis and Westlaw research tools. Advances in machine learning enhance these capabilities into something beyond a basic Westlaw search. For example, machine learning may be used to analyze the responsive documents identified based on the parameters initially inputted to determine other keywords, time frames, custodians, or other parameters not previously identified that will result in identifying additional responsive documents. This process is called “predictive coding” and is frequently used in review and production of electronically stored documents, especially e-mail. Research conducted as early as 2006 indicated that predictive coding identifies more than 95 percent of relevant, responsive documents compared with slightly better than 50 percent identified by manual review.7 Courts have explicitly recognized the efficiency of predictive coding, and permitted parties to utilize predictive coding in searching for and collecting responsive documents, even over the objections of the other party.8 This efficient review is not limited to written documents either. AI natural language audio processing has been used to review telephone calls in Telephone Consumer Protection Act cases filtering 33,000 hours of audio recordings into 140 hours of relevant data to be reviewed by a human.9

Long gone are the days of sitting in a warehouse searching through hundreds of boxes of documents. In the near future, if not already, the notion of running a search and spending hours manually filtering through the results will be viewed as quaint and inefficient.

Specific Uses for AI

In recent years, a variety of services have been offered to attorneys that are AI-based. A few of these systems are described here.

Patent and Trademark Searching

In the world of patent litigation, searching for prior art (i.e., preexisting technologies and documents that may render a patent invalid) has been compared to “looking for a needle that did not already exist in an ever-growing haystack.”10 For this reason, several companies have created AI systems designed to identify relevant prior art so that inventors can assess the patentability of their inventions. Such systems can help companies determine whether they can safely enter a particular technology space without (or at least with less) fear of patent infringement lawsuits.

The U.S. Patent and Trademark Office (USPTO) has also recently sought to improve its patent search capabilities with AI.11 In pursuing this goal, it issued a request seeking expertise in AI stating that it “is interested in the use of advanced technology to supplement and improve USPTO search capabilities with differentiating solutions that utilize technological advances.”12 The USPTO specifically wants a system that will review and analyze patent disclosures to then identify prior art that is relevant to the reviewed disclosure.

Similarly, several companies are offering AI products in the field of assisted trademark searching and policing. Some companies claim to use machine learning and natural language analysis to quickly determine if a mark is able to be protected. Such products also enable a party to police its mark, evaluating potential infringers, and monitor the trademarks of its competitors.

Contract Review

Contract review fits within the wheelhouse for AI systems as well. Large companies can have hundreds if not thousands of contracts executed per year, whether with suppliers, vendors, contractors, or customers. Reviewing each of these contracts is routine (sometimes mind-numbingly), slow, labor-intensive, and expensive. AI systems have been released that are designed to check for inconsistencies and errors in and among a company’s contracts that humans may miss. One notable company claims that its AI is able to flag contract conditions that are unmet, assess discrepancies between contracts for performance and actual performance, identify nonstandard contract terms and clauses, maintain consistent sales terms among customers, and simplify the contracting process.13 AI contract review also has the capability of reviewing for the absence of clauses or terms (i.e., what is not there), as well as evaluating the contract itself.

The AI systems claim to be not only more accurate than human review, but also much more time efficient. For example, one startup contends that its system can review and approve contracts in a fraction of the time it would take a person to perform the same task. Indeed, the company reported testing which determined that the AI reviewed and identified issues in five sample nondisclosure agreements at an accuracy of 94 percent compared with an average rate of 85 percent for experienced lawyers.14

Automation

AI is also being used to automate what were previously manual processes. As one example, TurboPatent automatically generates patent applications based on disclosures provided by an applicant. Its AI claims to forecast possible grounds for rejection and art units to which the application will be assigned in an effort to mitigate these risks in the drafting process. Its related product SmartShell generates draft responses to USPTO office actions based on the office action, claims, and application. SmartShell has packages to generate technical and legal arguments in response to the grounds for rejection cited by the USPTO.

AI is also being integrated into translation services. TransPerfect, a large translation company, has invested heavily in AI-based machine translators that are designed to mimic natural language using artificial neural networks. Machine translators are significantly faster and cheaper than human translators. Depending on the purpose for the translation, AI-based translations are frequently an adequate replacement for human translators, despite less accurate and more awkward translations.

Statistics

One of AI’s greatest strengths is its ability to process and analyze large amounts of data. Lex Machina, a legal analytics system, has collected thousands of gigabytes of data on court rulings, case resolutions, and damages awards, among other things, in a variety of federal court practice areas. The Lex Machina system then uses natural language processing to extract the information contained within the court decisions and machine learning to analyze the data. From this data, the Lex Machina system can provide analyses about factors such as time to trial, likelihood of success on early motions to dismiss, and likelihood that a certain party or counsel will settle a case. A similar company called Premonition claims that it is able to predict which cases will settle, which will go to trial, and who will be the prevailing party if a case does go to trial. A different system from Casetext called CARA looks for trends in opposing counsel’s arguments based on past arguments to assess how the attorney will present his or her arguments in court.

Limitations of AI

Despite all of these advantages, AI still has substantial shortfalls. First, and most glaring, the AI is only as strong as the algorithm and data underpinning it. The strength and weakness of AI is that it does exactly what it is programmed to do. A human must program the AI system at the outset and must supervise and review the results produced by the AI system. Failure to adequately supervise can result in malpractice allegations. In J-M Manufacturing Co. v. McDermott Will & Emery,15 a law firm was sued and accused of failing to adequately supervise, among other things, a discovery vendor that used search term and keyword filters to filter privileged documents. Unfortunately for the firm, it did not adequately review the vendor and check the documents before production, which resulted in privileged documents being produced in a qui tam action by the government. The case is still ongoing seven years later and serves as a reminder to not put complete faith in the AI systems utilized.

AI also lacks human reasoning that can provide a “sanity check” to the results and lacks the personal experience that leads to a person’s intuitive response to situations. This reasoning and experience can allow a human lawyer to evaluate merits and likelihood of success. This also applies to AI systems that have a stylistic component, such as translation. A recent study published in the MIT Technology Review determined that, while AI translations work well at the word or sentence level, translations of whole documents still leave much to be desired.16 In the comparison of human to machine translations, the fluency of the translation, which is effectively a measure of how awkward the translation is, was significantly better for human translators than for machines. The accuracy of the translation at the document level was also better for human translators.

Additionally, human lawyers still maintain the advantage in oral advocacy. Even in this realm, however, people’s advantage may be slipping. This past summer, IBM exhibited its “Project Debater” AI that is designed to make coherent arguments against a human opponent.17 In a live debate against two human opponents, the audience voted a draw after the AI system conveyed more information to the audience than its human opponent and was more persuasive, despite having a worse delivery.

Finally, overreliance on AI carries inherent risks. Our legal systems were built by humans, with human problems in mind. Just as it can be jarring to hear a computerized voice, because something in the inflection is not “quite right,” so, too, can AI results be not quite right. This is especially true because, in the end, it will be judges and juries—or in the case of business decisions, CEOs and board members—who will make the final decisions on given legal issues. Presently, at least, AI is a useful tool that is especially trustworthy in certain applications, but it should not be the alpha and omega of decision-making.

Potential Future Impact/Implications

Artificial intelligence will continue to improve, while time and expense pressures on billing attorneys will increase. The efficiencies offered by AI free up human attorneys’ time to perform more complicated analytical tasks such as oral and written advocacy and negotiation. Leveraging AI systems will complement attorneys’ skills and value to clients by making the attorneys both better informed and more efficient. The AI systems will also allow for processing large amounts of data and discovery more efficiently with less errors, which will result in streamlining litigation and minimizing the impact that electronic discovery has brought into civil litigation. Finally, human attorneys will continue to be necessary to monitor the AI as a backstop to protect clients. Even the ABA Model Rules of Professional Conduct note that technology has its benefits and risks—malpractice is possible when relying too heavily on AI and not checking the work.18 It will therefore be interesting to see how humans comingle with AI systems in the coming years.

Endnotes

1. Thomson Reuters, Legal Department 2025: Ready or Not: Artificial Intelligence and Corporate Legal Departments 11 (2017), https://static.legalsolutions.thomsonreuters.com/static/pdf/S045344_final.pdf.

2. Daniela Hernandez & Ted Greenwald, IBM Has a Watson Dilemma, Wall St. J., Aug. 11, 2018.

3. Alan Ohnsman, Waymo Shifts to “Industrializing” Self-Driving Tech as Robotaxi Launch Nears, Forbes (Sept. 6, 2018), https://www.forbes.com/sites/alanohnsman/2018/09/06/waymo-shifts-to-industrializing-self-driving-tech-as-robotaxi-launch-nears/.

4. David C. Blair & M.E. Maron, An Evaluation of Retrieval Effectiveness for a Full-Text Document-Retrieval System, 28 Comm. ACM 289 (1985).

5. Daniel Martin Katz et al., A General Approach for Predicting the Behavior of the Supreme Court of the United States, 12 PLoS ONE, no. 4, Apr. 12, 2017.

6. Theodore W. Ruger et al., The Supreme Court Forecasting Project: Legal and Political Science Approaches to Predicting Supreme Court Decisionmaking, 104 Colum. L. Rev. 1150, 1150 (2004).

7. Jyothi K. Vinjumur, Predictive Coding Techniques with Manual Review to Identify Privileged Documents in E-Discovery 3 (2018) (unpublished Ph.D. dissertation, University of Maryland), https://drum.lib.umd.edu/bitstream/handle/1903/21009/Vinjumur_umd_0117E_19016.pdf?sequence=1&isAllowed=y.

8. Aurora Coop. Elevator Co. v. Aventine Renewable Energy - Aurora W., LLC, No. 4:12-cv-230, 2015 U.S. Dist. LEXIS 177331, at *3–4 (D. Neb. Jan. 6, 2015); Fed. Hous. Fin. Agency v. HSBC N. Am. Holdings, Inc., No. 11-cv-6189, 2014 U.S. Dist. LEXIS 19156, at *32–33 (S.D.N.Y. Feb. 14, 2014); Da Silva Moore v. Publicis Groupe SA, No. 11-cv-1279, 2012 U.S. Dist. LEXIS 58742, at *5–6 (S.D.N.Y. Apr. 26, 2012).

9. Michael McDonald, Artificial Intelligence Can Reduce % of Review Hours, L. Tech. Today (Sept. 6, 2017), https://www.lawtechnologytoday.org/2017/09/artificial-intelligence-can-reduce-review-hours.

10. USPTO’s Challenge to Improve Patent Search with Artificial Intelligence, Solicitation No. RFI-USPTO-AI-PATENT-SEACH-18, at 2 (Sept. 13, 2018), https://www.fbo.gov/index?s=opportunity&mode=form&id=8d661556b876e94883b69093a2531b91&tab=core&_cview=0.

11. Id.

12. Id.

13. AI for Contract Governance, IBM Watson, https://www.ibm.com/watson/contract-governance/ (last visited Dec. 11, 2018).

14. Artificial Intelligence More Accurate Than Lawyers for Reviewing Contracts, New Study Reveals, PR Newswire, Feb. 26, 2018, https://www.prnewswire.com/news-releases/artificial-intelligence-more-accurate-than-lawyers-for-reviewing-contracts-new-study-reveals-300603781.html.

15. No. BC 462 832 (Cal. Super. Ct. July 28, 2011).

16. Human Translators Are Still on Top—for Now, MIT Tech. Rev. (Sept. 5, 2018), https://www.technologyreview.com/s/611957/human-translators-are-still-on-top-for-now/.

17. Olivia Solon, Man 1, Machine 1: Landmark Debate between AI and Humans Ends in Draw, Guardian, June 18, 2018, https://www.theguardian.com/technology/2018/jun/18/artificial-intelligence-ibm-debate-project-debater.

18. Model Rules of Prof’l Conduct r. 1.1 cmt. (Am. Bar Ass’n 1983).

Timothy J. Carroll is a partner at Dentons US LLP in Chicago and a member of the Intellectual Property and Technology Group. He specializes in litigating patent dispute cases involving cutting-edge technology, as well as trade secret disputes.

Manny Caixeiro is a partner in the Los Angeles office of Dentons. He has extensive experience litigating commercial and patent disputes.

 

The authors would like to extend a special thanks to Nicholas Jackson.

Entity:
Topic: