ChatGPT, OpenAI’s revolutionary artificial intelligence chatbot, can significantly reduce the work I do in a day. By typing a prompt into OpenAI’s website, I can ask the chatbot to analyze and answer questions. ChatGPT will produce much more sophisticated answers than any previous chatbot. Instead of using Westlaw to look up a legal issue, I can ask ChatGPT to both look up the issue and generate an answer. Instead of writing a motion for summary judgment from whole cloth, I can get a first draft from ChatGPT. However, the reason I have said no to ChatGPT is twofold: First, I am not confident ChatGPT will give me an answer I can trust; and, second, at this early stage of my career, I could stand to learn things the old-fashioned way.
Yann LeCun, chief artificial intelligence (AI) scientist at Meta and professor at New York University, puts ChatGPT’s shortcomings succinctly:
[Large language models like ChatGPT] can pass the bar exam, medical licensing & MBA exams. But on the IIT entrance exams they perform badly on chemistry, horribly on physics, and terribly on math. They are good with rote learning & fluency but bad with building mental models & reasoning with them.
ChatGPT’s aptitude for mistakes is well documented. While ChatGPT will produce well-worded answers, the substance of the answers may be misleading or wrong. Earlier this year, these wrong but nice-sounding answers, often-called “hallucinations,” prompted the Attorneys’ Liability Assurance Society, a mutual insurance carrier, to issue a newsletter [subscription or payment required] to policyholders, warning about the potential risks of these hallucinations, as well as data privacy risks.
Confidentiality concerns loom over ChatGPT usage as well. ChatGPT is a large language model—an AI, machine-learning model trained on massive amounts of data to perform tasks like generating text in a conversational manner. When it was introduced, ChatGPT raised serious ethical concerns about attorney-client privilege and confidentiality. Though there are ways to deploy ChatGPT to alleviate concerns about inputting sensitive information (for instance, ChatGPT can be deployed outside of the original OpenAI platform under certain circumstances not discussed in this article, such as those discussed here and here), the default behavior of ChatGPT in OpenAI lacks the safeguards to guarantee absolute preservation of information covered by attorney-client privilege.
At this point in my career, the juice from ChatGPT does not feel quite worth its squeeze. While ChatGPT will give me a decently worded answer to my prompt, it does not save me any research time. I do not always know enough about the substantive law to know whether I can trust what ChatGPT produced. I still have to check cites and diagnose whether ChatGPT is accurate or risk being seduced by an AI chatbot designed to produce an answer it thinks I will like. Obviously then, the better I know the subject matter of the prompt, the better I can diagnose ChatGPT’s accuracy. ChatGPT gives me very little incentive to take the shortcut it offers. That feels especially true because, depending on how I use Chat GPT, confidentiality concerns could still exist. At the end of the day, when I take the time and do the work myself, I can be sure my bar license and my clients’ livelihoods are safe.
The biggest reason I have held off on shortcutting to ChatGPT, however, is that I am still learning to sew. I have work to do to understand how the practice of law actually works. When I think about the successful attorneys I work with, the core competency they share is that they know the law in their respective practice areas. They learned that law through sweat and study. Today they can issue-spot, correct inaccuracies, and command oral argument. They put in the work and time on the front ends of their careers to practice today with surgical skill. The same will be expected of me. What I cannot do is rob myself of understanding just because I want a quick answer. I am best served spending hours poring over code, learning rules, and gaining an overall understanding of their interplay—because when a judge asks me a question in open court, I cannot excuse myself to consult ChatGPT for the answer. I have to know it for myself. At least for now, the best path to that knowledge comes from continuing to master the hand stitch.
ChatGPT likely will not always feel worth avoiding. In fact, my ethical obligation to maintain competence under Model Rule of Professional Conduct 1.1 may one day require that I embrace it. Technology is rapidly advancing and correcting, and for that matter, so am I. It is only a matter of time before ChatGPT and large language models are fully incorporated into the systems we interact with on a daily basis. By that time, I hope, ChatGPT and I will be ready for each other.