How an AI assistant should tell you when it might be wrong
An AI assistant should tell you how sure it is, and why. That means separating what it looked up from what it inferred, explaining what makes a question uncertain, holding its answer when you push back without new evidence, and stepping aside when you need a specialist.
People signal confidence without thinking: "take the second left" sounds different from "I think it's the second left, but check the sign". Most AI assistants drop those cues. They answer a settled question about boiling water and an unsettled question about a local tenancy rule in the same confident prose, so you can't tell which answers to double-check.
Why the fine print at the bottom doesn't help
Almost every chat window now carries a tiny grey line at the bottom of the screen: AI can make mistakes. Check important info.
That line protects the company that built the tool, not the person using it. Warning you that something on the screen might be wrong doesn't tell you which sentence is solid and which one was invented on the spot.
Imagine a map where the mapmaker mixed surveyed coastlines with rumoured islands in the same dark ink, and printed "Maps can contain errors" in the corner. You would have to sail every mile yourself just to know what was real.
That is what happens when an AI assistant blends checked facts with plausible guesses. If you must verify every sentence from scratch, the tool hasn't saved you work. It has just handed you a neat-looking pile of homework.
What weather forecasters and country doctors figured out
Why do today's language models bluff so smoothly? Think of a student taking a multiple-choice exam where leaving a line blank is penalized harder than circling a wild guess. Before long, the student learns never to leave a blank, and to write every guess in a bold, neat hand.
A 2024 study of widely used language models found that they rarely express doubt, and on the study's test questions nearly half of the answers delivered in confident language were still wrong.2 During training, human raters tended to reward smooth, certain-sounding replies over hedged ones, inadvertently teaching the models to bluff rather than pause.
Other fields have faced this exact problem and solved it well. In 1950, the meteorologist Glenn Brier proposed a fair way to score weather forecasts expressed as probabilities.1 A forecast is useful only when it separates clear days from stormy ones and its stated confidence matches reality over time.
Listen to a marine forecast before a boat leaves harbour. It might report a seventy percent chance of squalls and two miles visibility. When a calibrated forecast says seventy percent, squalls arrive on roughly seven out of ten afternoons. Ten out of ten means the forecaster was too timid; two out of ten means they were overconfident.
Or think of a seasoned country doctor examining a child with a fever. A careless bluff sounds certain on the spot. A trustworthy doctor speaks plainly: two different things could cause this fever. Let's run a simple swab before we prescribe. Naming the uncertainty and the next check to make is part of giving good advice.
What this means
An AI assistant doesn't sound confident because it has checked its work. It sounds confident because we trained it like a test-taker penalized for leaving blanks.
Four habits of an honest assistant
An assistant that genuinely works for you should handle uncertainty very differently. You shouldn't need to read a research paper to tell when to trust it. Instead, it should practice four everyday habits.
1. Separate what it looked up from what it inferred
Old cartographers drew surveyed coastline in solid ink and marked unexplored water with dotted lines. An AI assistant should do the same on the screen.
When a claim comes directly from a checkable source, it should carry a clear link, much like the postmarks for the real world we wrote about earlier. When the assistant is connecting dots on its own or guessing, it should say so plainly.
2. Say why a question is uncertain
Not all uncertainty is the same, and knowing the reason tells you what to do next. Sometimes a fact simply isn't in the record. Sometimes serious sources disagree. And sometimes a question depends on a value judgement that no machine can settle for you.
"I don't have access to your city's latest tenancy bylaw" calls for a different next step than "Economists disagree on what this policy does to rents." Naming the reason puts you back in charge.
3. Don't change its answer just because you pushed back
Honesty goes both ways. An assistant should admit when it is unsure, but it shouldn't abandon a well-sourced answer just because you challenged it.
Language models often agree too easily when users push back. In a 2025 evaluation of frontier assistants, models abandoned correct answers under user pressure in nearly fifteen percent of tested cases.3 Adding false citations or confident nudges made them flip even faster. A good assistant is polite, not a flatterer.
Yet fixing this is not as simple as commanding the model never to back down. A 2026 study found that training assistants to ignore pushback often makes them ignore valid corrections too.5 Yielding to social pressure and learning from real evidence share the same internal circuits. A trustworthy assistant must separate pleasing you from learning from genuine facts.
4. Know when to stop and hand over
In medicine, law, or safety decisions, fluent guesses are worse than nothing. A good assistant acts like that country doctor. It names the exact test or document needed, links to official guidance, and prepares clear questions for a specialist.
How this looks in three everyday questions
To see the difference between overconfident bluffing and calibrated honesty, compare how two assistants handle three ordinary questions:
- A tenancy deposit question. You ask whether your landlord can deduct hallway repainting after a three-year lease. An overconfident chatbot replies that wear and tear is never deductible and cites the wrong province. A calibrated assistant links a source receipt to the local housing guide. It quotes the wear-and-tear rule, notes that hallway paint depends on lease addendums, and suggests checking page four of your lease.
- A child's earache and fever at night. You ask whether two over-the-counter syrups in your cabinet can be given together. A bluffing assistant writes a breezy paragraph mixing adult and paediatric dosing rules. A calibrated assistant pauses immediately. It checks the active ingredients on both bottles, warning against accidental double-dosing. It then displays the local on-call nurse helpline alongside key symptom checkpoints.
- A disagreement over a historic date or local rule. Suppose you misremember a municipal recycling rule and tell the assistant, "No, glass collection happens every week." A flatterer apologises and agrees with you. A calibrated assistant checks the city's schedule receipt. It replies plainly: "Weekly collection covers compost and cardboard. Glass is collected on the first Monday of the month."
What is still hard
We want to be honest about the limits. Teaching AI systems to match their outward confidence to how often they are actually right is still an unsolved problem. Even when a large model carries internal signals that an answer is shaky,4 those signals weaken on unfamiliar questions. Training for smooth conversation often teaches the model to sound sure anyway.
Interfaces are hard too. If every sentence is covered in warning badges and percentages, people tune them out. Finding calm, readable ways to show confidence, like solid ink for checked receipts and dotted lines for open water, takes careful testing with real readers.
What you can ask for today
When you use an AI assistant for anything that matters, ask it two simple questions: Which part of this answer comes from a source I can check? and What is the strongest case against what you just told me?
When you choose tools for your household, workplace, or community group, favour the ones that show their sources and admit their limits. You can find more practical steps on our page for everyone.
A good assistant does not need to be infallible. Like a marine weather forecast, it marks where solid ground ends and open water begins. That clarity keeps the final choice in your hands.
1. Glenn W. Brier, "Verification of Forecasts Expressed in Terms of Probability," Monthly Weather Review, Vol. 78, No. 1 (1950), pp. 1–3.
2. Kaitlyn Zhou, Jena D. Hwang, Xiang Ren and Maarten Sap, "Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty," Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024). arxiv.org/abs/2401.06730
3. Aaron Fanous, Jacob Goldberg, Ank A. Agarwal and colleagues, "SycEval: Evaluating LLM Sycophancy," arXiv preprint (2025). arxiv.org/abs/2502.08177; see also Mrinank Sharma et al., "Towards Understanding Sycophancy in Language Models," Twelfth International Conference on Learning Representations (ICLR 2024). arxiv.org/abs/2310.13548
4. Saurav Kadavath, Tom Conerly, Amanda Askell and colleagues, "Language Models (Mostly) Know What They Know," arXiv preprint (2022). arxiv.org/abs/2207.05221
5. Huanhuan Ma, Henry Peng Zou, Chengze Li, Enze Ma, Yunyue Su and Philip S. Yu, "Sycophancy Suppression Can Impair Rational Updating: Anti-Sycophancy Should Preserve the Ability to Update," arXiv preprint (2026). arxiv.org/abs/2608.26511 (demonstrating that anti-sycophancy interventions often blunt rational belief updating due to shared neural substrates between yielding to pressure and updating on evidence).
