When AI Makes Up Case Law: The Hidden Risk Every Law Firm Must Understand

August 14, 2026

6–9 minutes

1,339 words

Artificial intelligence is transforming legal practice.Legal research can be faster. Document review can be accelerated. Drafting can become more efficient. Large volumes of information can be analysed in a fraction of the time previously required.

But there is a serious problem that every law firm adopting AI needs to understand:AI can be confidently wrong.

And in legal practice, confident misinformation is not a minor inconvenience.It can become a professional liability problem.

What Is an AI Hallucination?

An AI hallucination occurs when an AI system generates information that appears plausible but is factually incorrect or entirely fabricated.The problem is particularly dangerous in legal work because AI-generated text can look authoritative.

A fabricated citation may have:

a realistic case name;
a plausible court;
an apparently correct citation format;
convincing legal language;
and an argument that sounds completely reasonable.

The output can look exactly like something a lawyer would expect to see.Except the case may not exist.

When AI Invents the Law

Imagine asking an AI system:”Find cases supporting this argument.”

The system produces five authorities.They look perfect.The lawyer incorporates them into a submission.

The document is filed.Then opposing counsel checks the citations.One case does not exist.Another case exists, but the quoted passage does not.A third citation points to a real decision, but the decision says the opposite of what the AI claimed.The problem is no longer technological.

It has entered the courtroom.

The Four Risks Lawyers Need to Understand

AI hallucinations in legal practice can take several forms.

1. Fabricated cases

The system creates a case that never existed.

2. Fabricated quotations

The case exists, but the quotation attributed to the judgment does not.

3. Incorrect legal authorities

A real case or statute is cited, but it does not actually support the proposition for which it is being used.

4. False legal analysis

The citations may even appear legitimate while the AI’s interpretation of the law is incorrect.

These failures can be particularly difficult to identify because the output often sounds confident.

The Confidence Problem

This is one of the most dangerous characteristics of generative AI.A human lawyer who is uncertain may say:”I need to check this.”

An AI system can produce:”The court held that…”

with complete confidence.The language itself provides no reliable indication of whether the underlying proposition is correct.That creates a dangerous psychological trap.The better the AI becomes at producing fluent legal language, the easier it can become to mistake fluency for authority.

But:

Fluency is not accuracy.

Confidence is not verification.

A citation is not evidence that the cited authority exists.

AI Does Not Replace the Lawyer’s Duty to Verify

This is where professional judgment becomes essential.An AI-generated legal answer should be treated as a starting point, not as a final legal authority.Every important proposition needs appropriate verification.

That means checking:

whether the case actually exists;
whether the citation is correct;
whether the quoted language appears in the judgment;
whether the case is still good law;
whether the court actually decided the proposition claimed;
and whether the authority applies to the relevant jurisdiction and facts.

The AI can accelerate the research.The lawyer remains responsible for the research.

Why This Is an AI Governance Issue

It is tempting to treat hallucinations simply as a technical limitation.That is too narrow.

The real question for a law firm is:What controls exist to prevent an AI error from becoming professional work product?

That turns hallucination into a governance issue.A mature AI governance framework should establish:

Human review

AI-generated legal content should receive appropriate human review before it becomes client advice or court-facing work.

Citation verification

Legal authorities should be independently verified before being relied upon or filed.

Approved tools

Firms should understand which AI systems lawyers are permitted to use for legal work.

Internal policies

Employees should know what AI can and cannot be used for.

Training

Lawyers and staff need to understand hallucinations, model limitations, confidentiality risks, and verification requirements.

Auditability

Where appropriate, firms should maintain sufficient records to understand how AI was used in significant workflows.

The Real Question Is Not “Can AI Do Legal Research?”

It clearly can assist with legal research.

The better question is:“What role should AI play in legal research?”

There is an enormous difference between asking AI to:identify potentially relevant authorities

and asking AI to:determine which authorities should be relied upon without verification.

The first can accelerate a lawyer’s work.The second risks delegating professional judgment to a system that does not understand the consequences of being wrong.

The Human-in-the-Loop Principle

The answer is not to prohibit AI from legal practice.That would ignore the enormous potential benefits.

The better approach is to establish human-in-the-loop controls.

AI can:

→ identify potentially relevant cases;

→ summarise documents;

→ organise large bodies of information;

→ generate preliminary research questions;

→ produce a first draft;

→ highlight possible issues.

The lawyer then:

→ verifies the authorities;

→ checks the underlying sources;

→ evaluates the reasoning;

→ considers jurisdiction and context;

→ applies professional judgment;

→ and takes responsibility for the final work.

That is augmentation.

Not abdication.

Should Lawyers Disclose AI Use?

This raises a more difficult question.Should courts require lawyers to disclose when AI has been used to prepare legal filings?There is no simple answer.Disclosure could promote transparency and encourage more responsible use.But disclosure alone does not solve the underlying problem.A filing does not become reliable simply because the lawyer says AI was involved.The more important requirement is verification and accountability.A lawyer who uses AI responsibly and verifies every authority may produce excellent work.

A lawyer who does not use AI at all can still file an incorrect citation.

The central issue is therefore not simply:“Was AI used?”

It is:“Was the work properly verified?”

AI Makes Verification More Important, Not Less

There is an important paradox here.The more powerful AI becomes, the more useful it becomes for legal professionals.But the more convincing its outputs become, the more important verification becomes as well.A crude AI system producing obviously poor answers is relatively easy to distrust.A sophisticated system producing elegant, plausible, occasionally fabricated legal analysis is much more dangerous.

The problem is not that AI is always wrong.The problem is that you may not know when it is wrong without checking.

The Law Firm AI Policy Every Firm Needs

Before deploying generative AI for legal work, firms should establish clear answers to several questions:

Which AI tools are approved?

What information may lawyers enter?

What information is prohibited?

Which tasks may be automated?

Which tasks require mandatory human review?

How must legal citations be verified?

Who is responsible for the final output?

What happens when an AI-generated error is discovered?

How is AI use documented for high-risk workflows?

These questions turn AI adoption from informal experimentation into controlled professional practice.

The Competitive Advantage Is Responsible Adoption

The firms that benefit most from AI will not necessarily be those that use it most aggressively.They will be the firms that understand its limitations as well as its capabilities.They will use AI where it creates genuine value.They will establish controls where the risks are significant.They will train lawyers rather than simply giving them access to software.

And they will preserve human judgment where professional responsibility demands it.

The goal is not:

AI instead of lawyers.

It is:

AI assisting lawyers while lawyers remain accountable for the work.

Conclusion

AI can accelerate legal research.It can analyse documents.It can generate drafts.It can identify potentially relevant authorities.But it can also invent a case that never existed.It can misquote a judgment.It can misunderstand a statute.It can produce an argument that sounds authoritative while being legally wrong.

That is why every AI-generated legal output should begin with one assumption:It may be wrong.

The responsible lawyer does not reject AI because it can hallucinate.The responsible lawyer builds a system in which a hallucination is caught before it becomes legal advice, a court filing, or a client decision.AI should accelerate legal work.

It should never replace legal judgment.And in the age of generative AI, the ability to verify what the machine says may become just as important as the ability to ask it the right question.

← Back to All Blogs