Deepfake detection improves by learning new tricks without forgetting old ones
Learning as Deepfakes Evolve: RF-Prompt for Continual Audio Deepfake Detection
SoundArtificial Intelligence
Summary
Detecting fake speech is tricky because new fake-speech tricks keep appearing, and detectors must remember how to catch old ones too. The authors studied this problem carefully by controlling different challenges, like changing real speech types versus new faking methods. They proposed a smart way called RF-Prompt that keeps good knowledge about real speech and adds new knowledge for each new fake method, mixing them in a flexible way without needing to know which fake it is during testing. Their method worked better than others on a realistic test setup that reflects how new fakes really arrive over time.
What this means in practice
- •For security software engineers: Build audio deepfake detection tools that keep improving with new fake types while remembering previous ones without performance loss.
- •For voice authentication teams: Enhance voice verification systems to resist evolving audio forgery methods by integrating a model that adapts incrementally to new attacks.
Authors
Yuankun Xie, Xiaoxuan Guo, Xiaopeng Wang, Siqing Qin, Shaole Li, Kong Aik Lee
Abstract
Continual audio deepfake detection requires learning newly emerging deepfake methods while retaining discrimination of previously encountered speech. Existing dataset-incremental evaluation changes both real-speech domains and deepfake mechanisms, making their effects difficult to distinguish. We construct five task organizations over identical training, development, and evaluation pools to study these factors under a controlled sample budget. Our proposed Real-Anchored Mechanism-Incremental (RAMI) protocol reflects the practical setting in which available real speech provides a recurring mixed-domain reference while new deepfake mechanisms arrive incrementally. We further propose RF-Prompt, an asymmetric continual prompt-learning method that preserves reusable real-speech knowledge through a shared real prompt and expands mechanism-specific knowledge through inherited fake experts with orthogonal residuals. Input-adaptive soft fusion combines the accumulated experts into a fixed number of injected tokens without requiring task identity at inference. On RAMI, RF-Prompt achieves 10.110% average EER and 10.370% pooled EER, outperforming all evaluated continual-learning baselines. Across the five controlled protocols, RAMI yields the lowest common-average and pooled EER. Component ablations, limited-data experiments, and cross-backbone evaluations further validate the proposed design.