A five-minute check turns a fluent answer into a checked one

Start here

A chat answer arrives fully formed, with no visible seam between what a source said and what the model added. The task is not judging whether an assistant is good in general, but whether this answer holds up before it goes into an email, a report or a decision. Asked whether a person can trust the AI to tell the truth, OpenAI's help page says the system 'can occasionally produce incorrect answers' and recommends checking responses. Anthropic's documentation for builders names the same problem: models 'can sometimes generate text that is factually incorrect or inconsistent with the given context.'

What the documents say

Anthropic's guide lists techniques for lowering that risk: letting a model say it does not know, asking it to pull a direct quote before answering, asking it to verify each claim against a supporting quote and retract what it cannot support, and asking it to reason step by step. A reader can run these by hand in an ordinary chat window. A study with 80 crowdworkers compared fact-checking with a search engine against fact-checking with a language model's explanation. The model group worked faster with similar accuracy, but participants over-relied on it when its explanation was wrong, trusting a fluent wrong answer more than messy search results. Both supporting and opposing arguments together reduced, but did not remove, that over-reliance. Fluency is not evidence; the failure to watch for is a convincing wrong answer, not an obviously bad one.

Check this

This is an editorial checklist built from the documents above, not a claim it has been tested here. Ask the assistant for the source behind a claim, and open it. Compare one concrete number, date or quote in the answer against what that source says. Ask the assistant to argue the opposite conclusion; a model that builds an equally confident case against its own answer suggests the first was not well grounded. Ask directly whether it is certain, since Anthropic's guide notes that permission to express uncertainty changes what a model reports. Decide only after those steps. Citation and comparison are checkable in a way fluent prose is not; the opposite-case step mirrors the contrastive-information effect the study measured.

What holds and what fails

The check holds best on factual, checkable claims: a date, a statistic, a named source, a quoted policy. It holds less well on judgment calls or open questions with no single source to settle the matter, because there is nothing fixed to compare. It also depends on actually opening the source, since a citation that exists is not one that supports the sentence attached to it. This is an editorial extension of the cited documents, offered as a routine rather than a guarantee.

• Pick one recent chat answer taken at face value and ask for the source behind its central claim.
• Open that source and check one number or date before repeating the claim elsewhere.
• Ask the assistant to argue the opposite case and see whether the answer still holds.

None of this needs a paid tier, only the extra minutes the documents recommend before an answer counts as settled.