AI docs · Quality & evaluation
Hallucinations
When a model produces fluent, confident output that is wrong or fabricated. A core risk to design around.
What it is
- A hallucination is a plausible-sounding but incorrect or invented output.
- It happens because models predict likely text, not verified truth.
How it works
- Models fill gaps with what is statistically likely, which can be fabricated facts, citations, or details.
- They are more likely when the model lacks the needed information or is pushed beyond its knowledge.
Trade-offs
- You can reduce hallucinations (grounding, retrieval, constraints, verification) but not fully eliminate them.
- Mitigations add cost and complexity.
When to use it
- Always design for it in any system whose output people act on.
Common pitfalls
- Trusting confident output without verification, especially facts, numbers, and citations.
- Assuming RAG or a bigger model removes the risk entirely.