When an assistant reads the rules
Language models produce fluent text, not legal judgments. When public agencies use AI agents to check permits, benefits or filings, every decision should cite the specific statutory text it relied on, and the file should go to a human caseworker whenever rules conflict or the system is unsure.
An agent asked whether an applicant qualifies for a heating grant can produce a paragraph that sounds like the regulation without being grounded in it. A human inspector checks the measurement against the exact clause and, when two bylaws disagree, hands the file to someone with authority. Agents that decide public matters need the same discipline.
Fluent guessing
When a modern language model reads an official application, it does not consult a physical rulebook. It calculates probabilities across words. It predicts which phrases usually follow one another in government reports.
Most of the time, this produces answers that sound astonishingly polished. The assistant quotes administrative phrases with flawless grammar. It drafts formal denial letters that read like they came from a seasoned legal clerk. It cites section numbers with total confidence.
Yet under the hood, the model is guessing. If a regional statute changed three months ago, or if two clauses contain a delicate exception, a general language model often invents a plausible-sounding compromise. In public administration, this is dangerous. A machine does not need to sound angry to harm a family. A polite, fluent hallucination can deny a winter heating subsidy or block an accessible wheelchair ramp just as effectively as a broken furnace.
What this means
Fluency is not accuracy. A software assistant that writes like a lawyer is still guessing unless every sentence is mechanically anchored to an official legal text.
Grounding decisions in the statute
To prevent these harms, public bodies cannot deploy raw commercial chatbots as legal decision-makers. They need two concrete engineering boundaries.
First, the assistant must be bound to an authoritative statutory corpus.1 When a municipal agent evaluates a permit, it cannot pull legal interpretations from its training weights. It must retrieve the official text of the relevant bylaws directly from an authenticated public repository. Every conclusion must include an exact section citation, a verbatim excerpt, and a verifiable hash proving which version of the law was in effect that morning.
In one small evaluation, a compliance agent built this way separated every conforming text from every non-conforming one in a 15-text test set.1 When models are restricted to retrieving and comparing verified legal paragraphs, they stop hallucinating imaginary exemptions. The software acts less like an unguided author and more like the building inspector’s scale ruler.
When the algorithm must hand the file back
The second safeguard is even more important: mandatory human escalation.
No statutory code covers every human situation. Life is messy. A family applying for childcare relief might have a non-standard employment contract. A community centre renovating its roof might sit across two zoning boundaries.
In standard commercial software, an agent is rewarded for giving a final answer quickly. But in public administration, giving a fast answer is far less important than giving a fair one. When a compliance agent encounters conflicting bylaws or uncertainty above a defined threshold, it must not guess. It must stop immediately and hand the complete file to a human caseworker.1
This escalation cannot be optional. Public procurement rules should require automated systems to include deterministic stop switches. Whenever an applicant would be denied a vital public service, a human officer must personally review the evidence and sign the determination.2
This principle is already entering public policy. Guidelines from international standards bodies and public directives now mandate that automated systems provide clear human override controls and meaningful explanations for adverse decisions.3
Honest limits
We should be realistic about what technology can and cannot do. Binding an assistant to a statutory corpus prevents hallucinations, but it does not make bad laws fair.
If a city bylaw is written ambiguously, or if an administrative rule unfairly excludes migrant workers, software will simply enforce that injustice with mechanical precision. Maintaining verified digital repositories of municipal codes across thousands of jurisdictions is also an enormous civic challenge.
Technology cannot replace political accountability. It cannot relieve citizens of the duty to attend council meetings, question procurement budgets, and demand fair rules. But by demanding statutory grounding and human caseworker escalation, we can ensure that public algorithms remain humble tools rather than unaccountable judges.
You can explore how community leaders and public officials put these boundaries into practice in our Shared Horizon action library.
- Soukaina Alaoui Ismaili and Jean-Sebastien Dessureault, “An LLM-Based Agent for Legal Compliance Assessment: Architecture, Memory, and Evaluation in a Canadian and Quebec Context,” arXiv preprint arXiv:2610.27347 (2026).
- Treasury Board of Canada Secretariat, “Directive on Automated Decision-Making” (2024).
- European Parliament and Council, “Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act),” Official Journal of the European Union (2024).
