The problem
Retrieval-augmented systems still produce claims their sources don’t support. Detecting those hallucinations usually needs a trained classifier and labelled data.
How it works
A three-agent pipeline decomposes each response into factual claims, verifies every claim against the source passages, and flags unsupported spans using task-specific decision rules. It was evaluated on RAGTruth across QA, summarization and data-to-text.
The full write-up is on the research page.
03 /
Key decisions
- Zero-shot on purpose
No fine-tuning and no GPU, so the pipeline can be dropped into any RAG system without labelled data.
- Rules decide, agents verify
Agents extract and check claims; task-specific rules aggregate their verdicts into the final decision.
04 /
Results
- F1
- 0.642
- Records
- 600
- Training examples
- 0
- Per record
- ~$0.0003