Summary
It is November 1, 1851, in New York City. You are an average person of reasonable means, and—out of genuine necessity—you need to sew. You are neither a tailor nor a seamstress, but you know the basic functions of sewing: Thread goes through needle, needle pierces cloth, needle and thread weave in and out of cloth; tie a knot to finish.
You are on your way to the general store to purchase some cloth, needles, and thread, when a copy of the latest issue of Scientific American catches your eye. An avid follower of the latest engineering and mechanical developments, you stop to read. There, in volume 7, issue 7, is the solution to your sewing problem: Isaac Singer’s sewing machine.
The advertisement is not lost on you. This machine shortens the time needed to build a man’s shirt from 14 hours to a mere 1 hour and 16 minutes. You purchase a sewing machine and sit down to begin your sewing project. The only problem: You do not actually know how to use a sewing machine. This revolutionary machine is useless to anyone who does not know the first thing about puckering or staggered stitching. Before this tool can do you any good, you will have to learn the skill of sewing.
I have had my law license less than a year, and at this point in my practice, I feel akin to a novice sewer learning to stitch at the introduction of the sewing machine. Like most recent law school graduates, I know enough abstract legal principles to have passed the bar exam. Until about a year ago, however, I never applied any of them to a real-world problem. It turns out the practice of law is entirely different from the practice of theorizing about law. Knowing the answer deadline feels a lot like knowing generally that thread weaves through fabric. Both are a far cry from filing an answer or sewing a shirt. Still, a buzzy mainstream shortcut often tempts me to cut my task times in half.
ChatGPT, OpenAI’s revolutionary artificial intelligence chatbot, can significantly reduce the work I do in a day. By typing a prompt into OpenAI’s website, I can ask the chatbot to analyze and answer questions. ChatGPT will produce much more sophisticated answers than any previous chatbot. Instead of using Westlaw to look up a legal issue, I can ask ChatGPT to both look up the issue and generate an answer. Instead of writing a motion for summary judgment from whole cloth, I can get a first draft from ChatGPT. However, the reason I have said no to ChatGPT is twofold: First, I am not confident ChatGPT will give me an answer I can trust; and, second, at this early stage of my career, I could stand to learn things the old-fashioned way.
Yann LeCun, chief artificial intelligence (AI) scientist at Meta and professor at New York University, puts ChatGPT’s shortcomings succinctly:
[Large language models like ChatGPT] can pass the bar exam, medical licensing & MBA exams. But on the IIT entrance exams they perform badly on chemistry, horribly on physics, and terribly on math. They are good with rote learning & fluency but bad with building mental models & reasoning with them.
ChatGPT’s aptitude for mistakes is well documented. While ChatGPT will produce well-worded answers, the substance of the answers may be misleading or wrong. Earlier this year, these wrong but nice-sounding answers, often-called “hallucinations,” prompted the Attorneys’ Liability Assurance Society, a mutual insurance carrier, to issue a newsletter [subscription or payment required] to policyholders, warning about the potential risks of these hallucinations, as well as data privacy risks.
Confidentiality concerns loom over ChatGPT usage as well. ChatGPT is a large language model—an AI, machine-learning model trained on massive amounts of data to perform tasks like generating text in a conversational manner. When it was introduced, ChatGPT raised serious ethical concerns about attorney-client privilege and confidentiality. Though there are ways to deploy ChatGPT to alleviate concerns about inputting sensitive information (for instance, ChatGPT can be deployed outside of the original OpenAI platform under certain circumstances not discussed in this article, such as those discussed here and here), the default behavior of ChatGPT in OpenAI lacks the safeguards to guarantee absolute preservation of information covered by attorney-client privilege.
At this point in my career, the juice from ChatGPT does not feel quite worth its squeeze. While ChatGPT will give me a decently worded answer to my prompt, it does not save me any research time. I do not always know enough about the substantive law to know whether I can trust what ChatGPT produced. I still have to check cites and diagnose whether ChatGPT is accurate or risk being seduced by an AI chatbot designed to produce an answer it thinks I will like. Obviously then, the better I know the subject matter of the prompt, the better I can diagnose ChatGPT’s accuracy. ChatGPT gives me very little incentive to take the shortcut it offers. That feels especially true because, depending on how I use Chat GPT, confidentiality concerns could still exist. At the end of the day, when I take the time and do the work myself, I can be sure my bar license and my clients’ livelihoods are safe.
The biggest reason I have held off on shortcutting to ChatGPT, however, is that I am still learning to sew. I have work to do to understand how the practice of law actually works. When I think about the successful attorneys I work with, the core competency they share is that they know the law in their respective practice areas. They learned that law through sweat and study. Today they can issue-spot, correct inaccuracies, and command oral argument. They put in the work and time on the front ends of their careers to practice today with surgical skill. The same will be expected of me. What I cannot do is rob myself of understanding just because I want a quick answer. I am best served spending hours poring over code, learning rules, and gaining an overall understanding of their interplay—because when a judge asks me a question in open court, I cannot excuse myself to consult ChatGPT for the answer. I have to know it for myself. At least for now, the best path to that knowledge comes from continuing to master the hand stitch.
ChatGPT likely will not always feel worth avoiding. In fact, my ethical obligation to maintain competence under Model Rule of Professional Conduct 1.1 may one day require that I embrace it. Technology is rapidly advancing and correcting, and for that matter, so am I. It is only a matter of time before ChatGPT and large language models are fully incorporated into the systems we interact with on a daily basis. By that time, I hope, ChatGPT and I will be ready for each other.
