Multimodal models detect errors across images videos and audio

OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

Computation and LanguageComputer Vision and Pattern Recognition

Summary

Multimodal large language models, which handle images, videos, and sounds along with text, sometimes make mistakes called hallucinations where they say things that don’t match the input. The authors created OmniHallu, a system that can find these mistakes for lots of different tasks and types of media all in one framework. They also made a big test set called OmniHallu-Bench with human-checked examples to help evaluate how well their system works. Their approach breaks down model outputs into small claims, checks each with specialists for the related media, and then combines this information carefully to decide if there’s a hallucination. They also built a faster version that keeps accuracy while needing fewer checks.

What this means in practice

  • For multimodal ai developers: Detect and reduce hallucinations in multimodal AI outputs across images, video, and audio generation and understanding tasks using OmniHallu.
  • For automated content verification teams: Improve accuracy of automated systems that verify claims made by AI models working with media by using a unified framework verified on diverse modalities.

Authors

Jianjiang Yang, Peihang Li, Shanqing Xu, Mengchen Qian, Lu Zhang, Meng Luo

Abstract

While Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse tasks, they suffer from hallucinations where generated outputs contradict or misrepresent input semantics. Existing research typically addresses hallucination detection within a single modality or task type, limiting generalizability. We introduce OmniHallu, a unified hallucination detection framework spanning both comprehension and generation tasks across image, video, and audio modalities. We contribute OmniHallu-Bench, a 10,000-sample benchmark with claim-level human annotations covering six cross-modal tasks: image-to-text (I2T), video-to-text (V2T), audio-to-text (A2T), text-to-image (T2I), text-to-video (T2V), and text-to-audio (T2A). Our multi-agent architecture decomposes model outputs into atomic claims, verifies them through modality-specific experts, and aggregates evidence via structured reasoning. We further propose a preference-optimized trainable verifier that approximates the multi-agent decision boundary, reducing expert calls by 66% with minimal performance loss. Extensive experiments reveal a consistent modality-dependent performance gradient and provide fine-grained insights into cross-modal hallucination patterns.
View PDFOpen arXiv