RAG and Retrieval for Legal Teams

Legal work is a near-perfect fit for retrieval-augmented generation and a near-perfect trap for naive use of language models. The fit is obvious: legal questions are answered by finding and applying specific authority, which is a retrieval problem. The trap is equally obvious in hindsight — a model asked for authority it cannot retrieve will produce something that reads exactly like authority, and the profession has now accumulated a long public record of sanctions demonstrating what happens next.

This curriculum treats that failure mode as the design constraint that shapes everything else. The organising principle is simple and non-negotiable: every generated claim must resolve to a passage that a human can open and read. Systems that cannot guarantee this are not made acceptable by better prompting.

Why retrieval, specifically

The instinct when a model gives a wrong answer is to reach for a bigger or newer model. For legal work this is usually the wrong move, and module one spends substantial time on why.

A language model's parametric knowledge — what it absorbed during training — is unversioned, undated, and unattributable. It cannot tell you which edition of a statute it internalised, whether an authority has since been overturned, or where a particular proposition came from. None of those properties are acceptable in legal practice, and none of them improve meaningfully with model scale.

Retrieval changes the shape of the problem. The model is no longer being asked to know the law; it is being asked to read a passage that was fetched from a corpus you control, and to summarise or apply it. That is a task models are considerably better at, and more importantly it is a task whose output can be checked, because the source passage is right there.

Who this is for

Mixed groups work best: in-house counsel who will use the system, knowledge management staff who will curate the corpus, and legal ops or legal engineering people who will run it. The chunking decisions in module two have direct consequences for answer quality that only practising lawyers can evaluate, and the evaluation set in module three cannot be built without people who know what a correct answer looks like.

The two days in detail

Day one — architecture and corpus design

The morning makes the retrieval case and establishes data classification constraints. Before any technical decision, teams determine which document categories — client files, privileged material, internal advice, public authority — can be processed under which deployment model. This gates everything that follows.

The afternoon is the chunking workshop, and it is the most technically consequential session of the two days. Legal documents break badly under naive chunking: a clause severed from its definitions section becomes meaningless or actively misleading, a provision separated from its jurisdictional header can be silently applied to the wrong body of law, and a superseded version retrieved without its date is worse than no answer. Teams design a chunking and metadata scheme against their own actual document types and stress-test it against edge cases they bring from practice.

Day two — evaluation and citation discipline

The morning builds the evaluation set: sixty real questions, each paired with the specific passage or passages that correctly answer it, drawn from the organisation's own corpus. This is laborious and it is the point — it is also the only way to get a defensible measurement, and the resulting artefact is reusable against every future vendor or model change.

Critically, participants learn to measure retrieval and generation separately. When a system gives a bad answer, it failed in one of two distinct ways: it retrieved the wrong passages, or it retrieved the right passages and then summarised them badly. These have completely different fixes, and teams that measure only end-to-end answer quality cannot tell which happened.

The afternoon covers failure handling. A retrieval system that finds nothing relevant must say so. Teams write an explicit standard for insufficient-authority responses, confidence signalling, and the boundary between what the system answers and what it escalates to a human.

The jurisdiction and recency problem

Fabricated citations attract the headlines, but the more common and more insidious failure is correct authority applied wrongly — real law from the wrong jurisdiction, a genuine provision that has since been amended, or a case that remains good law on one point and was overturned on another.

No amount of generation-side prompting fixes this; it is a metadata problem, solved at ingestion time. Module two covers encoding jurisdiction, effective dates, supersession chains, and document status into retrieval metadata, and module three covers building evaluation cases that specifically probe for these failures rather than only testing the easy questions.

Related curricula

Teams building rather than buying should pair this with agent harness engineering; teams whose immediate need is everyday assistant usage should start with Claude for work.

Related reading

• Grounding, RAG, or fine-tuning: a decision guide
• What is MCP (Model Context Protocol)?
• How to read AI benchmarks
• AI and law: contracts and legal help

Sessions are delivered by explainx.ai and adapted to the organisation's corpus, practice areas, and jurisdictional scope.