Who Checks the Citations? Benchmarking Legal Hallucination Detection

The study presents a new dataset of 1,300 legal excerpts with fabricated citations and benchmarks five AI models, including GPT-5, for detecting these hallucinations. GPT-5 achieved 82.8% recall and a 60.5% F1 score in an agentic framework, but all models faced challenges with subtle errors and required an average of 16.9 verification steps per excerpt. This research highlights the limitations of current AI systems in legal contexts and emphasizes the need for improved citation checking tools, particularly for users without access to commercial legal databases.