HALAS: A Human-Annotated Dataset of Hallucinations of Modern ASR Systems

HALAS is a newly introduced human-annotated dataset designed to analyze hallucinations in end-to-end Automatic Speech Recognition (ASR) systems, derived from real earnings call recordings. It includes span-level labels and demonstrates that even ASR outputs with low Word Error Rates can exhibit hallucinations, with benchmark results showing character and semantic-level metrics achieving 81% ROC-AUC compared to only 53.1% F1 score for existing methods. This dataset provides a crucial resource for practitioners to develop more effective detection and mitigation strategies for hallucinations in ASR systems.