Years ago, a lawyer friend boasted that he had landed a cushy gig, abstracting loan documents and leases. He was paid per transaction: the more leases he reviewed, the more money he made. He would read all the documentation and then fill out a detailed questionnaire. At the time, it was common to outsource this work to the Philippines, but my friend’s clients found that the error rate of such services was so high that it was worth paying a trained American lawyer to do the work.
Today, this work is mainly done by artificial intelligence (AI) engines, trained to search for specific terms and extract key data from these leasing documents. Today, my lawyer friend’s role would be to train the AI to respond to the questionnaire accurately. This is accomplished by reviewing the outputs of the AI engine against the source lease or loan transaction and making corrections. Much as a senior attorney would train an associate or paralegal, this correction process would guide the AI engine as to what answers the attorney is looking for.
This continuous review and correction (training) distinguishes AI from typical software programming. Once an error is identified, the trainer supplies the correction, and the AI program rewrites the underlying rules. One could view the AI program as an apprentice. In time, the apprentice will become a master tradesman, but until then, the master tradesperson must monitor and oversee the apprentice. There is no software code to review in an AI program. As the trainer, you must educate the AI through trial, error, and correction. Any AI-based application requires a continuous quality control process to monitor the results periodically and make adjustments.
As a real estate attorney, your job involves more than just abstracting lease and loan documentation. You are called upon to draft and review various documents. You need to gather information about the parties to the transaction, including name, place of incorporation, domicile, and authorized signatories. Suppose there is an existing lease being modified or a loan being consolidated into a new loan. In that case, you want to use data from these transactional documents to prepare the new transaction. This scenario is where an AI application and a document assembly engine can be used to increase drafting efficiency and accuracy.
In this article, I will discuss preparing a workflow using artificial intelligence and automated drafting. This approach is platform and software-agnostic. AI platforms are in the early stages of development, but they all share common elements. They are prompt driven. You need to ask a question. They are input driven. You must provide specific inputs, e.g., a set of documents for the system to evaluate or a constraint limiting the system to evaluate a limited set of information. The AI also requires instructions on the output. What form do you want the output to be? You can request answers that are short and factual or long and elaborate. The output can be free-form text, an extract, or a list of comma-separated values (CSV). Finally, you can provide instructions as to the level of accuracy, sometimes called temperature. At a temperature of zero, the answers are exact and factual. As you proceed up the scale, the answers become more creative; the engine infers an answer based on a range of facts.
There is a whole cottage industry of products that leverage AI. In evaluating these products, you should find out what engine is behind the product you are licensing. ChatGPT and Microsoft’s Copilot use the engine built by OpenAI. Google’s engine, called Gemini, was built into the Google search engine. Other engines include Claudi AI, Perplexity AI, Meta’s LlaMA, and xAI’s Grok and Dojo. Unless you have an in-house developer, purchase an application-builder tool to help you design your workflow.
Start with the job requirements. Define what you want your application, or app, to do. For the purposes of this article, we want our app to generate a commercial lease. As the landlord’s attorney, we will be drafting the initial document. Typically, the broker or real estate agent fields inquiries for a property and sends the marketing officer a request for a lease. The request may be followed up with a proposed term sheet, which the marketing officer may generate. The agent may also provide additional documentation about the tenant and information about guarantors. The information is reviewed for accuracy, and a lease is generated. Our goal is to speed up the process and involve the lawyers only when required to address a novel issue in the lease negotiations.
Break your requirements into phases, and then work backward. The final product is the lease, so let’s start there. Start by taking your form lease and a handful of fully executed leases and feed them as inputs into your AI engine. Start with the executed leases if you don’t have a form lease. These are your inputs. Ask the AI engine to produce a model lease with named placeholders for key terms and a series of alternative provisions for topics like warranty, condition of the premises, tenant promotion funds, signage, etc. In your instructions, make clear that the first document input, your form lease, should be considered the standard and that the other document inputs of executed leases are the variations. This will inform the AI engine that the variations are either placeholders for variables or alternative provisions. At some point in the near future, you may be able to produce a template with markup code for the document assembly engine of your choice, be it XpressDox, HotDocs, Knackly, or PatternBuilder.
Make an inventory of all the named placeholders. Use a spreadsheet to track these placeholders. You will assign each placeholder a name and a prompt. These will be the variables used by the document assembly engine. Similarly, you will want a question and a rule for each set of alternative provisions. The question could be: “In what state is the property located?” The rules could be: “property located in New York” or “property located in Connecticut.”
As you would in building a document assembly template, you will want to organize the variables and questions based on subject matter. These groupings will become pages or tabs in your document assembly engine. If you were creating a document assembly application, you would focus on making the question or prompt clear enough for the average user to understand. You would assume the user was an attorney or paralegal with subject matter expertise in real estate law. Your questions would be simple and short.
But we are going a step further. We are not working with a person but with an AI tied to a large language model. We will ask the AI to review documents and extract the answers to each question you need to complete the lease. That means we must understand what documents or sources will contain the information required for the app. We need to understand how to phrase the question so that the AI knows where to find that information in the source documents and how to format the result to use with the template.
Let’s take a step backward. Each question we need answered is a separate process when working with AI. Think of AI as a puzzle. You must define the magic words to instruct the AI to retrieve your desired information. Before you dive in, think like a lawyer training a junior associate or paralegal. For each question, determine what documents the apprentice should review. Is it the request for lease? Or will the answer be found in the term sheet? Will any of these documents have details like corporate form, state of incorporation, notice address, mailing address, or principal officers needed for the document automation system? For each question, identify where the answers will likely be found.
Once you have identified where the answers will be found, you must discover where the answer will lie in those documents and how to frame a prompt to elicit the answer you seek. This will be a process of trial and error. Use a document like the request for lease as the input and ask, for example: “Specify the name of the tenant.” If the AI gets the correct answer from the source document, repeat the process with a different source document. If you don’t get the correct answer, adjust your question until you do. Once you get a prompt that works 95% of the time, record the prompt on your spreadsheet.
You may find that the term sheet is an excellent source of answers to many of your questions. But what happens if you don’t have a term sheet? Identify other types of documents you might have access to. Repeat the process outlined above. You may find an AI prompt that works on the term sheet won’t work on the request for lease. So, you should have a specific prompt for each source document type.
Repeat this process for each question or variable on your spreadsheet. While you “train” the AI, you also learn what prompts to give the engine to get the information you want. Sometimes, the solution is to alter the prompt. Other times, you can train the AI by telling it that it got the answer wrong and giving it the correct answer. Repeat this process enough times, and the AI may learn to start giving you the “correct” response. In this fashion, AI is different from typical rule-driven software. The goal is to obtain a consistent response to your prompt that you can rely on. Changing the prompts and training the AI is the best way to accomplish this result.
As you proceed through this process, you will discover which documents contain the answers to particular questions. Consider reorganizing your questions based on the source document. If you have a request for lease document, for example, group all those questions that can be sourced from the request for lease. Create a mini questionnaire where you ask all those questions and validate that you are getting the correct answer for all your questions. At this point, you should refine your prompts and continue training the AI. Confirm the engine’s accuracy by uploading several more versions of the request for lease until you are confident the questions are working.
When you are satisfied that you are extracting as much accurate information as possible from the first document, move on to the next document. Questions that work on one document may not work on the next document. In one document, the prospective tenant may be identified as “tenant,” whereas in another document, it may be the “requestor.” Understand how parties are identified for each document and adjust your prompts accordingly.
When building your AI ingestor app, consider how the user will be providing the document to the AI engine. If the input document is in your document management system (DMS), you may need to put in the document ID or provide a search to find the document. If the document is not in the DMS, you should allow the user to upload the document. Some information is available only in emails or from a web search. To accommodate that, you will want to allow the user to copy and paste the source information into a text box for the application to analyze. Some AI engines can only review small chunks of information. The user will need to have a tool that breaks the source document into digestible chunks.
At this point, you can build the document assembly app, including the build-out of the interview. Replace the placeholders in the template with variables tied to the questions in your interview. Frame the alternative provisions with rules tied to the questions in your interview. Run the document assembly application to ensure it produces the desired result.
Now, bring in the AI ingestor. With some platforms, the AI ingestor will write to a database or a CSV file. If this is the case, you will want to connect the document assembly app to that database or use a CSV file as its answer source. In other cases, the AI ingestor is integrated with the document assembly app. In both cases, you will want to run the AI ingestor and review the data that appears in the document assembly interview. If certain information is missing, you return to the AI Ingestor and refine your questions or, possibly, find new document sources to use. If the extracted information is wrong, you may need further training in your AI engine. The question should be clarified, or the engine may need more examples before it understands what information you are seeking.
Once you have a working flow, you will want to run benchmark tests. How fast does the whole process run? It may be that the AI ingestion process takes too long to run on demand. If so, you may need to come up with a batch process where a lower-cost resource uploads all the required documents. You might have a process where documents are identified by transaction and type and then run through an automated extraction process. The document drafter is then notified when the analysis is complete. At that time, the drafter can review the interviews, make any required corrections, and then produce the documents.
Though there is great potential for AI engines, don’t assume that you can subscribe to a platform and that it will work perfectly out of the box. AI engines should be treated like new hires. Like new associates or paralegals, the AI engine needs to prove its utility, and it will require training and oversight. Be judicious as to where in your practice you deploy the AI engine. Weigh the engine against the platform costs, plus the refinement, training, and oversight costs. As with any expense, weigh the return on investment (ROI). There are benefits from faster turnaround on document drafting, fewer emails back and forth with the client, fewer drafting errors, and more flexibility in your documents. Also, review your billing structure. Fixed fee billing or transaction-based pricing will allow you to quantify your ROI (profits before and after the app) and guarantee a faster return on your investment. AI, used properly, has a bright future in the legal industry.
