Time-Frequency Consistency Learning for Robust Speech Deepfake Detection
2026-07-20 • Sound
SoundArtificial Intelligence
AI summaryⓘ
The authors studied how real-world speech processing tools, like echo cancellation and noise suppression, mess up the ability to detect fake speech created by deepfake technology. They found that these processing steps distort the audio in ways that confuse current detection methods. To fix this, they created a new method called Time-Frequency Consistency Learning (TFCL) that helps the model recognize fake speech even after it has been changed by these tools. Their experiments show that TFCL keeps detection accurate despite these common audio changes.
Speech Deepfake DetectionAcoustic Front-End (AFE)Echo CancellationNoise SuppressionAutomatic Gain ControlVoice Activity Detection (VAD)Time-Frequency ConsistencyTemporal MisalignmentSpectral DistortionAttention Mechanism
Authors
Jun Xue, Zhuolin Yi, Yanzhen Ren, Yihuan Huang, Jiayu Xiong, Yi Chai, Guanxiang Feng, Jiajun Liu, Tong Zhang
Abstract
Recently, speech deepfake detection (SDD) has achieved significant progress. However, its robustness evaluation remains largely confined to controlled additive noise scenarios, lacking systematic investigation of the complex distortions introduced by acoustic front-end (AFE) processing pipelines in real-world deployments. In this work, we simulate a unified AFE pipeline comprising acoustic echo cancellation, noise suppression, automatic gain control, and voice activity detection (VAD), and conduct a comprehensive evaluation of current state-of-the-art models. The results show that the nonlinear and time-frequency coupled distortions introduced by AFE significantly degrade detection performance. To address this issue, we propose a Time-Frequency Consistency Learning (TFCL) framework, which aims to learn invariant spoofing representations that remain stable before and after AFE processing. We observe that AFE not only introduces temporal misalignment (e.g., segment-level shifts caused by VAD), but also weakens or distorts critical frequency-domain cues. To this end, TFCL employs an attention-driven soft alignment mechanism to capture cross-temporal dependencies, along with frequency-domain structural consistency constraints to enforce feature invariance. As a result, the model is able to maintain stable representations under both temporal perturbations and spectral distortions. Extensive experimental results demonstrate that the proposed method effectively mitigates the performance degradation caused by AFE processing, significantly improving the robustness of SDD in real-world scenarios. The code is available at https://github.com/JunXue-tech/TFCL.