How to check what an AI tells you
A few checks that catch most wrong answers: which claims to verify, how to ask for sources, and why a confident tone tells you nothing.
A model writes fluent, confident text whether or not it is right. Tone says nothing about accuracy, so the checking falls to you. Most of it takes a couple of minutes.
Know which claims are risky
Explanations of well-known ideas are usually fine. The risky claims are the specific ones: numbers, dates, names, quotations, citations, version numbers, prices, and anything describing a particular library, law or product. There a model is most likely to produce something that sounds right and is invented (see hallucination).
Ask which parts to verify
Skip is this true. Ask which parts of that answer you would need to verify and where you would look. The model cannot check itself, but it can usually point at the sentences worth checking, and that saves you rereading the whole answer.
Check the source
If the answer cites a paper, a web page or a court case, open it. Confirm it exists, says what the answer claims and carries the same numbers. A citation that does not exist is among the most common failures, and looking is the only way to catch it.
Test it on what you know
Before you rely on a model for a subject you do not know, ask it two or three questions in an area you know well. How it handles them shows how far to trust it on the rest, and whether it admits uncertainty when it should.
Get the important part a second way
Redo a calculation in a spreadsheet. Run code against a test. Find a second independent source for a fact. When two separate routes agree, you can lean on the result far more than on one confident answer.
Let it say it does not know
Add one sentence to the prompt: if you are not sure, or the answer is not in the text I gave you, say so and do not guess. The problem does not vanish. The balance tips toward an honest gap.
Notes from the author
Match the effort to the stakes. Skim-check what you will forget tomorrow. Verify properly anything you will send to a customer, publish or act on.
For information only, not professional advice. You use it at your own risk and we are not liable for any loss or damage. See the disclaimer.
Tagged