- Would You Trust a Lawyer Who Refused to Explain Their Advice?
- Law Is Built on Reasons
- The Rise of the Black Box
- Prediction Without Explanation
- Explainability Is More Than Transparency
- Accountability Requires Auditability
- The Regulatory Direction
- Questions Every Law Firm Should Ask
- Explainability Is a Principle of Justice
- Conclusion
- References
Would You Trust a Lawyer Who Refused to Explain Their Advice?
Imagine asking your lawyer whether you should settle a case.
They respond:
“There is a 70% chance you will lose.”
You ask why.
They refuse to explain.
No legal reasoning.
No discussion of the evidence.
No reference to precedent.
No analysis of the risks.
Just a number.
No client would accept that advice.
No court would consider it persuasive.
Yet many organisations are now purchasing AI systems that operate in exactly this way.
They generate recommendations.
They produce probabilities.
They assign risk scores.
But they cannot explain how those conclusions were reached.
In law, that is not merely a technical limitation.
It is a governance problem.
Law Is Built on Reasons
Legal systems do not simply produce outcomes.They justify them.A judge’s decision is not legitimate because of the result alone.It is legitimate because the reasoning can be examined, challenged, appealed, and defended.The written judgment is not administrative paperwork.It is the foundation of accountability.The same principle applies throughout the legal profession.Lawyers explain their advice.Arbitrators explain their awards.Regulators explain their decisions.
Every important legal conclusion is expected to rest on transparent reasoning.Reasoning is not an accessory to justice.It is justice in practice.
The Rise of the Black Box
Many advanced AI systems function as what computer scientists describe as black boxes.They receive large quantities of information.They process complex statistical relationships.They generate outputs with remarkable speed.But the internal reasoning often remains opaque.An AI litigation tool may conclude there is a 70% probability of losing a case.A contract review platform may identify a clause as high risk.A predictive sentencing model may assign a defendant a particular risk score.The recommendation may be accurate.
The difficulty lies elsewhere.Can anyone explain why?
Prediction Without Explanation
Artificial intelligence excels at prediction.Legal practice depends on justification.These are fundamentally different concepts.
Prediction answers:What is likely to happen?
Legal reasoning answers:Why should this conclusion be accepted?A probability without explanation is difficult to evaluate.It cannot easily be challenged.It cannot be independently verified.It cannot meaningfully support professional accountability.
In legal practice, unexplained recommendations should never become substitutes for reasoned analysis.
Explainability Is More Than Transparency
Explainability is often misunderstood as simply revealing the technical workings of an algorithm.For lawyers, the concept is broader.
Explainable AI should allow professionals to understand:
- which factors influenced the recommendation;
- how different variables affected the outcome;
- what assumptions the system relied upon;
- the confidence and limitations of the result;
- when human review becomes necessary.
This level of transparency enables meaningful professional judgment.Without it, lawyers become increasingly dependent on conclusions they cannot properly evaluate.
Accountability Requires Auditability
Professional responsibility does not disappear because software participates in legal work.If an AI-assisted recommendation contributes to negligent advice, the lawyer—not the algorithm—remains accountable.This creates an obvious governance question.How can professionals justify decisions if they cannot reconstruct the reasoning behind the tools they relied upon?
Responsible AI systems should therefore provide:
- comprehensive audit trails;
- documented decision pathways;
- identifiable sources;
- version histories;
- mechanisms for human review.
Without these safeguards, accountability becomes significantly more difficult.
The Regulatory Direction
Emerging AI regulation increasingly recognises this challenge.The European Union AI Act places particular emphasis on transparency, human oversight, technical documentation, record-keeping, and accountability for higher-risk AI systems.These requirements reflect a broader principle.AI should support human decision-making.It should never make accountability impossible.Legal professionals remain responsible for decisions affecting clients, courts, and society.
That responsibility requires systems capable of meaningful explanation.
Questions Every Law Firm Should Ask
Before adopting any AI system, firms should ask several fundamental questions.Can the vendor explain how the model reaches its conclusions?Can recommendations be independently reviewed?Is there an audit trail suitable for regulatory investigation?Can lawyers identify when the system may be wrong?Can outputs be challenged rather than simply accepted?
If the answer to these questions is unclear, the technology may introduce governance risks that outweigh its operational benefits.
Explainability Is a Principle of Justice
In The Quantum Guardian, I argue that explainability is not merely a technical feature.It is an ethical requirement.Human dignity requires decisions affecting individuals to be capable of justification.The rule of law depends upon reasons that can be understood, questioned, and evaluated.Artificial intelligence should strengthen those principles.Not weaken them.
A legal system governed by unexplained probabilities risks replacing reasoned judgment with statistical authority.
That would represent a profound shift in the nature of justice itself.
Conclusion
Artificial intelligence is becoming increasingly capable of supporting legal professionals.Its ability to analyse documents, identify patterns, and predict outcomes offers significant benefits.But legal practice has never been based solely on reaching the correct conclusion.It has always required explaining why that conclusion deserves acceptance.A lawyer who cannot justify their advice faces professional consequences.The same standard should apply to the technology that increasingly supports legal decision-making.In law, explainability is not a luxury.It is not an optional feature.It is the foundation of accountability.Because justice is not measured only by the decisions we reach.It is measured by our willingness to explain them.
References
- European Union. Regulation (EU) 2024/1689 – Artificial Intelligence Act.
- National Institute of Standards and Technology (NIST). AI Risk Management Framework.
- OECD. OECD AI Principles.
- UNESCO. Recommendation on the Ethics of Artificial Intelligence (2021).
- American Bar Association. Formal Opinion 512: Generative Artificial Intelligence Tools (2024).
- European Commission. Ethics Guidelines for Trustworthy AI (High-Level Expert Group on AI).
- Goudarzi, S. The Quantum Guardian.
- Goudarzi, S. AI for Legal Professionals.
- AI Adoption Starts with Trust, Not Technology
- Why AI Adoption Fails in Law Firms: The Middle Management Bottleneck
- The Leadership Gap: Why AI Resistance at the Top Puts Law Firms at Risk
- The AI Copyright Reckoning: Who Owns Creativity in the Age of Generative AI?
- AI Hallucinations Are No Longer a New Problem. They Are Becoming a Professional Responsibility Problem.