Sooner or later, a bot will tell you something false with a straight face. A made-up statistic. A quote nobody said. A product feature that doesn't exist. That's called a hallucination.
Why it happens
Remember what a model does: it predicts what a good answer looks like. See What is a model?.
Most of the time, the likely-looking answer is also the true one. But when the model doesn't actually know, it doesn't go silent. It produces the most plausible thing. A number that sounds right. A source title that fits the pattern.
It isn't lying. It has no intent. It's filling a gap.
Common gaps:
- After the cutoff. The model learned up to a date. Newer facts have to be looked up.
- Very specific details. Part numbers, prices, dates, exact quotes.
- Things only you know. Your customer, your process, your file names.
- Leading questions. "What year did X win the award?" can produce a year even if X never won.
Why confidence doesn't help
A person who's guessing often sounds unsure. A model usually doesn't. It writes a guess in the same calm voice as a fact. So tone tells you nothing.
The controls
You don't fix hallucinations with hope. You fix them like any quality problem: with controls.
| Control | What it looks like | ISO parallel |
|---|---|---|
| Give it the source | Hand it the document instead of asking from memory | Work from the controlled document |
| Require a source | "Every number needs a link and a date" | Evidence-based decision making |
| Let it say "I don't know" | Write "if you can't confirm it, leave it out" into the instructions | Nonconforming output is held, not shipped |
| Check before it ships | A person or tester bot verifies key facts | Inspection and release |
| Fix the rule | When one slips through, update the skill | Corrective action |
A real example
Our AN Creation bots carry a shared principle: never invent facts; trace every number to a dated source. In a review of 90 published bot templates, 54 had some version of that same rule in their text (our notes, good-practice.md). It's one of the most common rules in the library, because it's one of the most common failures.
This site runs on the same rule. Product facts are checked against xAI's docs. If we can't confirm something, it stays out.
What this means for you
- Ask for sources on anything that matters. Then click one.
- Make "I don't know" a safe answer. If your instructions punish gaps, you'll get filled gaps.
- Hand over the facts. A model reading your price list beats a model remembering prices.
- Scale the check to the risk. A brainstorm needs no check. A customer quote does. See Risk sets the rigor.
Hallucinations aren't a reason to avoid bots. They're a reason to manage them. See Don't make it up for the ISO side of this.