Deviation from Ground Truth! Can AI Be a Trained Legal Assistant?

Dec 7, 2024

Procurement of goods and services is an everyday task in large organizations.


This can range from essentials like toilet paper to complex professional services. Reviewing legal agreements for professional services or software licensing can be incredibly time-consuming—taking hours, if not days, for edits and suggestions.


All of that effort for only one contract.


Now, imagine managing 20 or 30 contracts at any given time. Add to this the back-and-forth negotiations between organizations and service providers, and it’s no surprise the process takes 3 to 4 months on average—or more than 6 months for complex deals.


“Can this process be streamlined using artificial intelligence (AI)?”


A Tedious Task for Legal Teams

I don’t know about you (and I’m not a lawyer), but reviewing legal documents isn’t my favorite pastime. Comparing two documents to look for risks and policy alignment is tricky.


Typically, one document outlines your company’s preferred terms and policies. For example:


  • "We should put all suppliers on a 90-day payment plan."

  • "We accept a minimum of 200% liability cap."

A legal assistant would compare these terms against a supplier’s agreement—often from major providers like Microsoft. The supplier’s document might have different fonts, formats, and structures. The assistant then highlights deviations and summarizes them for a senior colleague to assess the risks.


It’s a labor-intensive process. Can AI do it better?


From Cool AI to Value-Adding AI

Generative AI, like ChatGPT, has been making waves. It’s cool, exciting, and improving with every version. But how do we move from "cool and sexy AI" to "value-adding AI"?


For an AI legal assistant to make sense, it needs to:

  1. Ingest a legal agreement.

  2. Highlight deviations between the company’s policies and the supplier’s terms.

  3. Do this really fast.


If a legal assistant takes 40 hours to review and summarize an average agreement, AI should do it in less than 4 hours—a 10x speed-up. 


That’s value.


But this isn’t a task where you can just throw OpenAI’s ChatGPT at the problem. Your company’s policies aren’t in the public domain. The approach needs to be more sophisticated.


Using BERT to Create a Legal Assistant

The magic behind ChatGPT lies in Transformers, specifically a technique called Bidirectional Encoder Representations from Transformers (BERT).


In simple terms, BERT is a language model trained to understand language structure. It enables AI to accurately compare two sentences of different lengths with similar meanings.


Here’s how we used BERT to create an AI legal assistant:


  1. Encode the Organization’s Policies
    First, we took the organization’s favorable policies and encoded them into vectors using a fine-tuned proprietary BERT. These vectors became the reference terms.

  2. Compare Clauses
    Using a custom proprietary algorithm combined with BERT, we compared each reference clause against clauses in the supplier’s agreement.

  3. Generate a Deviation Document
    The AI produced a single Word document with two columns, comparing clauses from the two agreements—despite their differing authors, structures, and writing styles.


Real Benefits for Legal Practitioners

The benefits are massive:

  • The algorithm does the heavy lifting, generating deviation documents in about 25 minutes.

  • Legal teams can focus on scrutinizing high-risk clauses instead of slogging through the entire document.


A time-in-motion study with legal practitioners would quantify the value in hours and dollar terms—but the impact is already clear.


BERT isn’t just for legal use cases. It’s powerful in areas like:

  • Text classification (e.g., sentiment analysis).

  • Question answering.

  • Summarization.


Customized BERT models are already emerging for domains like biology, finance, and law.


Can We Build a Trained Legal Assistant?

The answer isn’t binary. The example above shows that AI and human collaboration is the best approach to solving this problem.


At Isazi, we often provide solutions that add real value, and for cases such as an AI-driven legal assistant, it's clear that AI and human collaboration is the next step to achieving this goal.


So, no need for lawyers to worry about AI... yet.