Custom software · Experimental project
Legal research: from a question to sources the lawyer can verify.
We are developing a tool for a single lawyer. They describe a legal question or a case, the system searches a pre-indexed corpus of case law and returns the relevant decisions with the exact passages. The lawyer verifies the result.
Framework
The framework of the experiment
These are the conditions under which the tool will be tested. They are not results: there are no measured results yet.
The problem
A convincing answer is not research.
A language model easily writes smooth text on a legal question. A lawyer needs something else: which decisions exist, what exactly they say and where they can be opened. That is why we do not optimise for a convincing-sounding answer, but for finding the important decisions and for sources that can be checked.
What we do not do
- Answers from memory: with an empty corpus or an unavailable source the system does not fill the gap with what the model “remembers”.
- Repaired quotations: the model may not fix a quotation or invent a decision number or an address.
- A chance of success: the ranking score is not a probability of the outcome of a case.
- Ready-made labels: “settled” or “binding” case law is not asserted without verified support.
What we do
- Decisions found: the case law on the specific question and facts, including the unfavourable.
- An exact passage: a verbatim quotation linked to the original decision.
- Visible limits: what the corpus covers, what is missing and what is uncertain.
- The lawyer’s judgement: the summary is a draft until the lawyer has reviewed it.
How it works
From the question to the summary
The search runs on a pre-built local index. The system does not fire dozens of requests at external sites for every research task.
A question or a case
The lawyer describes the legal question, the facts, the position represented and the procedural stage.
Breaking the question down
The question is split into a few essential sub-questions. Keywords, exact references to provisions and factual constraints are extracted. The original wording is kept.
Keyword and semantic search
The system searches the indexed decisions by exact wording and by meaning at the same time, with filters on decision metadata: court, chamber, type, period. The two lists are merged and passages are grouped by decision, so that ten excerpts from one ruling do not fill the whole first page.
Ranking for the case
A limited set of candidates is ranked against the specific question and facts: a similar legal issue, material facts, procedural stage and the issue actually decided. Irrelevant decisions are not added to make up numbers.
Exact passages and source
Each decision comes with its full text, a link to the original and the exact passages the conclusion relies on. If only a description is available for a decision, without full text, that is stated explicitly.
A short summary to verify
A summary with sources, stated uncertainty and search coverage. The lawyer assesses it and decides.
Quotations
How a quotation is verified
The check is mechanical: it compares text with text and does not depend on the model’s opinion. A quotation that fails it does not enter as a confirmed finding.
The source exists
The version of the decision the quotation comes from is stored and does not change.
The text matches verbatim
The quotation is compared with the stated span. A material difference makes it invalid.
It comes from the full text
A quotation from a short snippet in a results list is not accepted.
It belongs to this research
The decision and the place in it are among the sources found for the current question.
A text match does not prove the interpretation is right. That judgement stays with the lawyer.
What the summary contains
The summary is a short document with a fixed structure. The model supplies structured data and the application assembles the text, so an invalid quotation cannot appear as confirmed. It can be exported to a file.
- The question, the case, the position and the date.
- The key decisions found and why they are relevant.
- The exact passages with links to the original.
- Contrary case law and the material differences.
- The verified statutory texts and the date their wording is checked as of.
- Missing facts, unresolved references and uncertain conclusions.
- Corpus coverage, the filters applied and decisions without full text.
- The status: draft or reviewed by a lawyer.
Scope
What is in the first experiment and what is planned
A local application for one lawyer, with no public access and no unnecessary modules. The first version tests one thing: does the system find the important case law for a real question. That is why it starts with search only. Working with documents comes after that.
Planned after the first search test
- Cases and documents: saved cases and upload of text-based files.
- Facts: fact extraction with the place in the document cited and a human review.
- Case law for and against: analysis of supporting, contrary and distinguishable decisions.
- Administrative case law: if the selected cases require it.
- Historical versions of the law: only where a need is proven.
In the first experiment
- Corpus: a limited, pre-indexed set of court decisions in one selected area.
- Search: by keywords and by meaning, with filters on decision metadata.
- Quotations: exact passages linked to the original decision.
- Statute: the current, verified wording of a few acts for the selected area, with a stated date.
- Summary: short, with sources, with export.
- Measurement: lawyer feedback, time and cost per research task.
Evaluation
How we will know whether it works
Five real questions
One lawyer provides five anonymised questions and the decisions they know matter for them. The search is tuned on these.
A test with several lawyers
Three to five lawyers, with about ten real questions each. Some questions are held out and the search is not tuned on them.
Comparison with the current way
The lawyer solves the same task with the tool they use today and with the new one. Key decisions found and missed are recorded, along with the time, including verification.
A decision on hard thresholds
Zero fabricated decisions and zero invalid quotations presented as verified. Every critical miss is described. Without useful search and time saved there is no expansion.
Phases
Each phase has a condition for the next
Approach
We do not optimise for a convincing-sounding answer. We optimise for finding the important decisions and for sources that can be checked. That is why evaluation starts early: real questions from lawyers, compared with how they work today.
The system is a single module with one database: FastAPI, PostgreSQL with pgvector and a Python process for background jobs.
The application runs only on the lawyer’s trusted machine and is not exposed to the internet. Files stay in a private directory. Before real documents are submitted, the privacy terms of the language-model provider are checked.
Current stage
This is an experimental project. There are no measured results yet and we do not claim proven legal reliability. Success is measured by a test with lawyers on real questions, not by a working interface.
The tool supports research and does not replace the lawyer’s judgement.
What is not yet confirmed:
- The starting area of law. It is chosen together with the lawyer.
- The terms under which decisions from each source may be collected and stored.
- The source and licence for the statutory texts.
- The models for semantic search and the actual cost per research task.
Questions
Questions about the tool
Does the tool replace the lawyer?
No. The tool supports research: it finds decisions and shows the passages. Interpretation and the decision belong to the lawyer, and the summary is a draft until they have reviewed it.
Can the system make up a court decision?
The rules are built against it. Every quotation is compared verbatim with the text of the stored decision, and an invalid quotation does not enter as a confirmed finding. With an empty corpus the system does not answer from memory. Whether these rules hold in practice will be shown by the test with lawyers; there are no measured results yet.
Where do the court decisions come from?
From a pre-built local index in one selected area. For each source we first check what is technically possible and under what terms decisions may be collected and stored. A site being public does not mean permission for bulk download. Protections and restrictions are not bypassed; if collection is not permitted, what remains is manual import, or the phase stops.
Does it work for every area of law?
No. The first experiment is in one starting area, chosen with the lawyer. Corpus coverage is shown in the summary, and a missing decision is reported separately from a search miss.
Where are case documents kept?
The first version works only with the question and the case description. Document upload is planned for a later phase. The application is local, on the lawyer’s machine, with no public access.
Will this become a product for law firms?
Not for now. A multi-user version is a separate decision, taken only if the test shows the tool finds useful case law and saves one lawyer time.
Does your work depend on searching documents?
Tell us what you look for every day and where. We will discuss what can be indexed and how to verify the benefit.