Audio deepfake traces can change after voice model tuning
GenTraceBench: A Benchmark for Tracing Audio Deepfakes Across Pre- and Post-training Stages
Sound
Summary
Audio deepfakes use computer voices that are often modified after their initial training, which can affect how well we can detect their origin. The authors created a big test set to see if unique fingerprints of voice models remain after these changes. They found that some kinds of adjustments keep these fingerprints intact, but others can make tracing much harder. Their work helps improve detecting and verifying whether audio comes from a specific voice model, even after it is tuned.
What this means in practice
- •For cybersecurity teams: Detect and verify audio deepfakes even when the voice model has been altered after training.
- •For voice assistant developers: Build systems that recognize changes in voice models to improve authenticity checks on synthesized speech.
Authors
Li Wang, Kunyu Feng, Wan Lin, Dekun Chen, Qinke Ni, Xueyao Zhang, Lei Wang, Jie Shi, Haizhou Li, Zhizheng Wu
Abstract
Modern text-to-speech (TTS) systems are rarely deployed as unchanged pre-trained models. They are often adapted through supervised fine-tuning (SFT) or preference optimization such as DPO and GRPO. This raises a practical question for audio deepfake forensics: do fingerprints learned from a foundation generator remain valid after adaptation? We present GenTraceBench, a controlled benchmark spanning five TTS architectures, 16 pre-/post-training variants, and 49,728 utterances generated with fixed texts and speaker prompts. Under a train-on-foundation, test-on-adapted protocol, we evaluate binary detection, closed-set attribution, and open-set verification. DPO and GRPO generally preserve fingerprints, whereas some SFT and pre-training-data changes cause substantial drift; effect sizes vary across three forensic backbones. Repeated training runs confirm the largest W2V-BERT attribution drop, while a data-mixture control with comparable speech quality shows that composition change need not cause drift. In W2V-BERT verification, multi-shot enrollment reduces EER for the SFT condition from 44.4% to 11.0%, whereas the SingNet-only condition remains at or above 45% EER.