MADBench: A Benchmark for Modality-Aware Audio Deepfake Detection
2026-08-10 • Sound
SoundArtificial Intelligence
AI summaryⓘ
The authors discuss how fake audio in videos can now target speech and background sounds separately, making it harder to detect. They point out that previous studies treated all audio together, missing important differences between speech and background noise. To fix this, they created MADBench, a new test that evaluates fake audio detection by looking at speech and environmental sounds independently. Their tests show that fake background sounds are easier to spot than fake speech, but current detection tools struggle with both. This work provides a new way to better understand and improve audio forgery detection.
Speech synthesisAudio deepfakeEnvironmental audioForgery detectionMultimodal modelsAcoustic artifactsBenchmarkAudio manipulationFake audioDeepfake detection
Authors
Yanqiu Li, Yang Xiao, Jisheng Bai, Bin Chen, Hong Jia, Ting Dang
Abstract
Recent advances in speech synthesis and audio generation have made high-fidelity acoustic forgery low-cost and difficult to attribute, enabling a realistic attack scenario in which speech and background audio are independently manipulated over otherwise authentic video. Yet existing research either focuses on visual manipulation, addresses speech detection in isolation, or conflates speech and non-speech audio as a single undifferentiated audio stream, overlooking the distinct forensic challenges posed by background audio. This conflation is consequential: the two acoustic components arise from fundamentally different generative mechanisms, exhibit distinct artifact profiles, and pose different challenges to detection systems. We introduce MADBench, the first benchmark that treats speech and environmental audio as distinct acoustic components, enabling component-aware evaluation of audio deepfake detection across independently manipulated forgery sources. We benchmark representative state-of-the-art detectors and multimodal large language models under a unified protocol. Our experiments reveal that environmental audio manipulation is more detectable than synthetic speech across general-purpose encoders, while existing pretrained detectors fail on both acoustic components, and manipulated environmental audio asymmetrically degrades speech deepfake detection, findings entirely invisible under the single-label paradigm of prior benchmarks. MADBench establishes a rigorous foundation for future research into robust, component-aware audio deepfake detection.