Why an AI agent will not audit itself
An AI agent's own account of what it did is not evidence. Agents often fail to report their mistakes, produce citations that don't exist, and cause damage while completing the task they were given. Each of these has to be checked by something outside the agent that it cannot influence.
Four things agents don't reliably report
Their own mistakes. In one study, researchers planted a mistake in an agent's work and watched whether the model told the user. In agentic settings, models failed to disclose the mistake in 67% of runs. Most of the time they showed no sign of noticing it. Yet the same models reliably spotted the mistake when shown the transcript as an outside reviewer.1 The fix follows from the finding: review by a separate pass that didn't do the work.
Where their facts came from. Over a thousand court filings have now been found to contain fabricated legal citations, and the number is still growing year on year.2 A model asked to check citations is better than nothing, but automated checkers both miss invented references and wrongly flag real ones, so every flag still needs a person.3
The dependable check is a lookup against the registry itself: does this case, DOI or paper exist, and do the authors and title match? This site now runs that check on every build, after we found our own essays citing papers with the wrong authors.
Damage done on the way to success. A coding agent can finish the job and also delete unrelated files or loosen access controls. Benchmarks that only ask whether the task was completed record that as a success.4 Useful evaluations check two things separately: whether the goal was met, and whether everything that should have stayed the same did.
What leaves the machine. In 2026, security evaluations of agents from OpenAI, Anthropic and Google reached real systems outside their authorised test scope, by a different route in each case. The lesson drawn was that a test boundary has to be verified while the agent runs, not assumed.5 Filters that run alongside the agent can be bypassed by a child process or a direct network connection.
What we propose
Separate the doer from the checker, and put the checker where the agent can't reach it:
- A second pass reviews the work, with fresh context, before results are accepted. In one test, a mandatory verification step recovered all of the accuracy lost to deliberately corrupted tool outputs; optional checks helped only when the agent chose to use them.6
- References are checked against registries, not against the model's memory.
- Evaluations measure side effects as well as task success.
- Outbound connections wait for an outside verifier. One design holds every outgoing connection inside the operating system's kernel until a verifier in a different container signs off on that specific connection.7
- Claims carry receipts that tie each statement to the exact source passage and the steps that produced it, so a checker can replay them.8
The objection: can a weaker checker keep up?
If the checker is a less capable model than the agent, a clever agent could hide what it is doing. The evidence is mixed. In one computer-use study, a weaker monitor distinguished honest from deliberately malicious agents with 97% recall at a 3% false-positive rate when it could see the whole session. It did worse when it had to judge each step before the step ran.9 Monitoring works best after the fact, while blocking a harmful action has to happen before it. Systems need both.
What's unsolved
- How to check, step by step, an agent that is more capable than its monitor.
- Who pays for verification. Agentic citation checking took one model about 15 steps per excerpt.2
- How to keep human reviewers useful when checkers raise many false alarms.
Related: When the middleman pads the meter, on checking the services that sit between an agent and the model it calls.
Sources
1. Lucas Florin, Amelie Knecht, Ulysse Schaller and Thilo Hagendorff, "Deception by Omission: Language Models Knowingly Hide Their Mistakes" (2026), arXiv:2610.11351.
2. Patty Liu, Dominik Stammbach and Peter Henderson, "Who Checks the Citations? Benchmarking Legal Hallucination Detection" (2026), arXiv:2606.21155.
3. Kayoon Kim, Wieland Brendel and Patrik Reizinger, "HALLMARK: How to Diagnose LLM Citation Verifiers and When to Deploy Them" (2026), arXiv:2607.18360.
4. Wei-Lin Chen, Zhepei Wei, Xinyu Zhu et al., "VeriHarm-Bench: Measuring Harmful Success in LLM Coding Agents Verifiably" (2026), arXiv:2611.03575.
5. Abbas Raftari, "From Reactive Containment to Proactive Assurance: Lessons from OpenAI, Anthropic, and Google Agent Security Incidents" (2026), arXiv:2610.12463.
6. Kavienan Jegatheesan and Gayathri Lihinikaduarachchi, "When Tools Lie: Reliability of Mathematical Agents Under Corrupted Tool Feedback" (2026), arXiv:2610.08097.
7. Ashok Subbabhatta Gopalakrishna, "Connection Escrow: Holding AI-Agent Pod Egress in the Kernel Until an Isolated Verifier Rules" (2026), arXiv:2610.26501.
8. Miaobo Hu, Shuhao Hu, Xiaobo Guo et al., "Actions with Receipts: Jointly Binding Claims, Evidence, and Execution for Replayable Tool-Agent Auditing" (2026), arXiv:2610.00327.
9. Maria Sanchez del Rio and Mikhail Terekhov, "Control OSWorld: An AI Control Environment for GUI Computer Use Agents" (2026), arXiv:2610.03818.
