Four hundred thousand contracts, finally answerable

The problem

A large enterprise held roughly four hundred thousand executed agreements across a document management system, three shared drives, and two acquired companies' archives. When a question arrived — how many contracts have uncapped liability, which agreements have change-of-control clauses, what is our exposure to this jurisdiction — answering it meant a team of paralegals reading contracts for weeks. Most questions therefore went unanswered, and risk was managed by assumption.

The hypothesis

Contracts are semi-structured. Clause types recur, and language varies within recognizable bounds. If clauses could be extracted, normalized, and indexed with the obligations they create, the corpus becomes a queryable dataset rather than a pile of PDFs.

The build

• Ingestion and OCR at scale — including scanned and photographed documents from the acquisitions, with quality scoring so unreliable extractions were flagged rather than trusted.
• Clause extraction and normalization — identifying clause types (liability, indemnity, termination, assignment, change of control, data protection, governing law) and normalizing them into structured attributes such as cap type, cap amount, notice period, and jurisdiction.
• Obligation register — the extracted commitments with dates and owners, so renewal windows and notice deadlines surface before they lapse rather than after.
• Grounded question answering — legal asks in natural language; every answer returns the contract, the clause, and the page. No answer is produced without a citation.

Rollout

We validated against a stratified sample reviewed by the legal team, measuring per-clause-type accuracy rather than a single aggregate number. Some clause types performed well immediately; others needed prompt and model work, and one — bespoke commercial terms in the older acquisition archive — was excluded until the corpus could be cleaned. Scope honesty here was more valuable than coverage.

Results

The corpus became searchable, exposure questions that used to take a quarter now take a day, and the obligation register caught renewal windows that had previously been discovered late. The legal team's posture shifted from reactive to informed.

What we'd tell you

• Report accuracy per clause type. A single aggregate number hides exactly the risk lawyers care about.
• Every answer needs a citation to the page. Legal will verify, and should.
• The obligation register often delivers more value than the search — deadlines are where money leaks.
• Exclude the corpus you cannot do well yet, and say so. Overstated coverage destroys trust permanently.