Large audio language models often produce ungrounded hallucinations
MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models
Machine Learning
Summary
Large audio-language models can talk fluently about sounds but sometimes say things that aren't true or aren't supported by what they hear. The authors separate these errors into two types: those not based on the audio itself and those not supported by real-world facts. They created MISHAP-Bench, a big test with 12,000 questions and audio clips, to measure these mistakes better. They tested ten top models, finding even the best still hallucinate a lot, and tried some fixes that only helped a bit. Their benchmark helps others check and improve these audio models.
What this means in practice
- •For audio software developers: Use MISHAP-Bench to test if audio-language models hallucinate when answering questions about sounds.
- •For voice assistant engineers: Evaluate and improve the factual grounding of voice assistants that analyze or describe audio input.
Authors
Zhi Wen Soi, Giulio Segalini, Jian-Jia Chen, Lydia Chen
Abstract
Large audio-language models (LALMs) produce fluent responses about audio but often hallucinate by making plausible yet ungrounded claims. Existing audio hallucination benchmarks mainly measure response correctness, leaving it unclear whether an LALM hallucinates or simply fails to understand the audio. We challenge correctness-based evaluation by defining two hallucination categories: (i) context, where claims are not grounded in the audio; and (ii) knowledge, where claims about audio-related topics lack support from externally verifiable facts. We introduce MISHAP-Bench, a comprehensive benchmark with 12,000 challenging open-ended question-audio pairs and a rigorous evaluation pipeline covering both categories. To evaluate open-ended responses, we propose a groundedness judge that uses reference rubrics and judge prompts guided by human annotations. We extensively evaluate ten state-of-the-art LALMs and show that hallucination remains substantial. Even a frontier model such as Gemini 3.7 Flash reaches a hallucination rate of 36.5%. We further adapt and benchmark four mitigation methods from multiple domains for LALMs. Despite some improvements, effective hallucination mitigation remains an open challenge. Finally, we call on the community to evaluate hallucination and benchmark mitigation methods with MISHAP-Bench.