Improved audio deepfake detection using dual-branch E-Branchformer model
Disentangled Global-Local Feature Learning with E-Branchformer for Audio Deepfake Detection
Sound
Summary
Voice deepfakes, which are fake audio that sounds like real people, can trick systems that rely on voice to verify identity. The authors created a new computer program that uses two separate methods working together to analyze both the overall context and small sound details in audio. They combined special techniques to better focus on important sound features to help spot fakes more accurately. Tests on several datasets show their approach works better than others, especially in real-world conditions. Their method also generalizes well, meaning it can detect new types of fake audio it hasn't seen before.
audio deepfakeself-supervised learningmulti-head self-attentionconvolutional processingdepthwise convolutionSqueeze-and-ExcitationE-Branchformerclass tokenequal error ratevoice synthesis
Authors
Phuong Tuan Dat, Ho Bao Thu, Nguyen Tran Trung, Pham Viet Hoang, Nguyen Thi Thu Trang
Abstract
The rapid advancement of voice synthesis technologies such as text-to-speech and voice conversion poses significant threats to speech-based authentication systems, necessitating robust deepfake detection methods. In this work, we propose a novel E-Branchformer-based architecture that effectively leverages self-supervised speech representations for audio deepfake detection. Our model employs parallel branches to simultaneously capture global contextual dependencies through multi-head self-attention and local temporal patterns through convolutional processing. To enhance discriminative capability, we integrate depthwise convolution and Squeeze-and-Excitation modules that enrich the classification token with refined patch token information after feature merging. Extensive experiments on ASVspoof 2021 LA, DF, and In-the-Wild datasets demonstrate state-of-the-art performance with equal error rates of 0.88%, 1.85%, and 6.30% respectively, substantially outperforming existing methods. Comprehensive ablation studies validate that the dual-branch architecture provides complementary discriminative information, Squeeze-and-Excitation Aggregation significantly improves SSL feature integration, and the combination of DWConv and SE modules is critical for effective class token enhancement. The superior performance on real-world scenarios demonstrates strong generalization capability to diverse acoustic conditions and unseen spoofing attacks.