Existing methods of accessing knowledge will coexist with generative AI

A natural question at this point might be: Wouldn’t having access to drafts and learning how things were edited help train the AI better? The answer is actually no, for several reasons.

For starters, these question/answer systems are based on technology that does not train the AI. Instead, the AI is provided with an accurate knowledge base that it can consult to answer the question. In other words, when asked a question, the AI looks for the correct document and then uses that document to formulate the answer. It does not rely on its internal knowledge. From this perspective, the AI should not be provided with incorrect or outdated information.

Additionally, even if we would go about retraining the AI with the knowledge base, it is important to understand that its learning process is different from how we humans learn. AI models need to be shown the correct example. Their learning process does not include a reasoning step that would help them understand the difference between a bad example and a good example.

Beyond limiting access to final versions, organizations might also want to restrict how far back to pull data. Likewise, there might be some documents that are stale because the legal or regulatory landscape has significantly evolved in recent years due to new rulings or other developments. For this reason, firms might want to limit the training data to a specific time range rather than stretching back years and years.

Note that determining what good content looks like isn’t a one-off endeavor. The definition of good content should be dynamic, and for precisely this reason, there should be an internal knowledge curation team in charge of maintaining the data that is used to train the model. It is, after all, an important part of the intellectual property of the firm.

Maintaining tradition

Grounding generative AI’s responses in good content is what unlocks its ability to serve as a useful tool—and really, the approach is not that dissimilar from the prep work required for traditional forms of knowledge management, whether it’s keyword search or templates.

Neither of those traditional forms of knowledge management just magically happens on its own. Work product needs to be structured and tagged—e.g., “this is an example of a commercial real estate lease,” “this is an example of a share purchase agreement,” and so on—and the best examples need to be identified.

This identification can occur through manual curation efforts or through more automated processes; for instance, the template downloaded most frequently from the document management system by members of a certain practice group would be identified as the best template to use as a starting point for a specific use case. Either way, the goal is to ensure valuable knowledge is identified so that it can be properly leveraged.

While generative AI is certainly a new arrow in the search-and-knowledge quiver that’s capable of unlocking new capabilities, traditional approaches are still going to be relevant. To ensure lawyers can get access to the knowledge they need to do their best work and deliver the best outcomes, firms would do well to make sure they’re not choosing one or the other—this is a case where old and new alike have something useful to offer.

Mind Your Business is a series of columns written by lawyers, legal professionals and others within the legal industry. The purpose of these columns is to offer practical guidance for attorneys on how to run their practices, provide information about the latest trends in legal technology and how it can help lawyers work more efficiently, and strategies for building a thriving business.

Interested in contributing a column? Send a query to [email protected].

This column reflects the opinions of the author and not necessarily the views of the ABA Journal—or the American Bar Association.