AI
What are LLM hallucinations and how do you reduce them?
LLM hallucinations are confident-sounding but factually incorrect outputs generated by a language model. They occur because LLMs predict statistically likely text rather than retrieving verified facts. You reduce them by grounding responses with retrieved source documents (RAG), constraining output with structured schemas, and implementing verification steps.
Key Considerations
- RAG with source citation is the most proven hallucination reduction technique in production
- Structured output (JSON mode, function calling) constrains the model to valid response shapes
- Lower temperature settings (0.0–0.3) reduce creative drift but don't eliminate factual errors
- For critical applications, add a verification layer: a second LLM call or rule-based check that validates claims against sources
- Monitor hallucination rates in production — log outputs and sample-audit regularly