Hallucination

A confident, fluent answer that is simply wrong - an invented citation, function, statistic or fact.

A hallucination is not a malfunction in the usual sense. The model produces the most plausible continuation, and a fabricated reference that fits the pattern of real references is highly plausible. Fluency is no signal of accuracy.

The cases that catch people out are specific and checkable: library functions that do not exist, papers with real-sounding authors, legal citations, version numbers, prices. Anything a model states with a number or a name deserves verification.

You can reduce the rate. Give the source material rather than relying on training data. Ask the model to mark what it is unsure about. Give it an explicit way to answer nothing - a prompt that demands three findings will get three findings whether or not they exist. Retrieval, described under rag, attacks the same problem from the data side.

Where this comes up

Prompts, configs and tutorials in the library that touch this term.