Audio reasoning benchmark reveals gaps in deepfake detection tools
SEAR: Spoofing Evidence-Grounded Audio Reasoning Benchmark for Audio Language Models
SoundMachine Learning
Summary
Detecting fake audio is tricky, and current tests mostly check if models give believable answers instead of verifying if they use real sound clues. The authors created SEAR, a set of tasks that tests whether audio language models can find and explain real evidence of tampering. They also built a method to help models gather and use sound evidence more effectively. Their tests showed many models struggle to truly reason with the sounds behind their decisions, but their new approach improved accuracy and explanations.
What this means in practice
- •For security software developers: Improve audio deepfake detection systems by incorporating tasks that verify models' use of real acoustic evidence.
- •For audio forensics teams: Use enhanced reasoning benchmarks to generate clearer forensic rationales explaining deepfake audio detection results.
Authors
Rong Wan, Suliu Qin, Jiaxi Li, Wei Xie, Wenwu Wang, Xiaolong Han, Lu Yin, Xilu Wang
Abstract
Audio language models (ALMs) are increasingly used for audio deepfake detection (ADD), yet existing benchmarks assess their verdicts or rationale plausibility without verifying the underlying acoustic evidence. To address this issue, we first introduce spoofing evidence-grounded audio reasoning (SEAR), a four-task AQA benchmark to evaluate ALM-based ADD through acoustic evidence identification and quantification, deepfake detection, and forensic rationale generation. We further propose a bona-fide-based acoustic evidence agent (BAEA), which equips a frozen ALM with controlled acoustic tools under \textsc{fixed} or \textsc{adaptive} evidence-acquisition policies. Experiments with six ALMs reveal a clear gap between plausible rationales and verifiable acoustic evidence reasoning, while BAEA-\textsc{Fixed} improves final verdicts and forensic rationales on both evaluation partitions. Controlled interventions further show that misleading evidence degrades both detection and grounding performance.