Acoustic models influence text-to-speech deepfake detection effectiveness
Tracing and Relearning Detection Evidence in Text-to-Speech Systems
Sound
Summary
Detecting fake speech generated by text-to-speech systems is important for security and trust. The authors studied which part of the speech generation process gives away clues that help detectors spot fakes. They found that changes in the acoustic model affect how easily a detector can tell real from synthetic speech. Even when the acoustic model was adjusted without trying to fool detectors, it became harder for fixed detectors to identify fakes. However, retraining detectors on the updated speech kept detection effective, showing that both acoustic model updates and detector adaptation are crucial for fake speech detection.
What this means in practice
- •For security teams: Enhance audio deepfake detectors by updating models to resist manipulation and adapting detectors to maintain detection on evolving synthetic speech.
- •For voice synthesis developers: Adjust acoustic models to reduce detectability of synthetic voices, enabling more natural and less detectable speech generation.
Authors
Eunji Shin, Kyudan Jung, Jihwan Kim, Minwoo Lee, Jaegul Choo
Abstract
Recent audio deepfake detectors separate bona fide speech from synthetic speech, yet it remains unclear which stage of a text-to-speech system supplies the detection evidence. We address this with controlled resynthesis and detector adaptation in an F5-TTS-BigVGAN pipeline. Since vocoder reconstruction of a real mel can itself be separable from the source utterance, we fix the vocoder and trace the larger change in detector separation to acoustic generation. Adversarially fine-tuning the acoustic model, with no detector in its objective, raises EER against fixed detectors at comparable quality. However, adapting a detector only on the tuned model's VCTK outputs lowers its LibriSpeech EER from 19.42% to 7.46% and improves detection of unseen base F5-TTS outputs. These results suggest that acoustic-model updates can reduce the detection evidence available to fixed detectors, while detector adaptation keeps the updated outputs detectable in this pipeline.