Replayfactor improves detection of replayed deepfake speech attacks
ReplayFactor: Replay-Aware Dual-Attribute Learning for Practical Speech Deepfake Detection
Sound
Summary
Deepfake speech that is played back with physical devices can fool many security systems. The authors designed a method called ReplayFactor that uses multiple learning techniques to better recognize these replayed fake voices. They also created a new large dataset of replayed fake Chinese speech to train and test detectors. Their approach significantly improves accuracy and reduces errors compared to older methods, helping systems detect these more realistic voice fakes.
What this means in practice
- •For voice security teams: Detect replayed deepfake audio attacks on access control or voice authentication systems more reliably using ReplayFactor.
- •For telecom fraud prevention teams: Identify and block synthetic replayed voice calls that attempt impersonation in telecom customer service or call centers.
Authors
Miao He, Jiayi Zhou, Yu Zheng, Zhongjie Ba, Peng Cheng, Yuxi Zhou, Junjie Zhu, Changhao Zhang, Jian Liu
Abstract
Advances in synthesis techniques have made deepfake speech increasingly realistic, amplifying voice security threats, including unauthorized voice cloning and impersonation. Although existing deepfake detectors perform well on digitally generated speech, their performance often degrades on physically replayed deepfakes, which constitute a low-cost and practical threat to real-world systems. Existing research focuses primarily on dataset construction and evaluation, while available dataset diversity and replay-aware detector design remain limited. We propose ReplayFactor, a dual-attribute framework that enhances replay evidence by fusing self-supervised learning (SSL) representations with an acoustic prior, while leveraging multi-task learning and supervised contrastive learning to promote robust, generalizable representations. We also construct ReplayFake-CN, a large-scale, multi-device Chinese dataset comprising approximately 185k replayed synthetic utterances to complement public benchmarks. Our framework improves macro-F1 from 60.99% to 76.44% and reduces EER from 11.86% to 3.05% on the EchoFake open set over the baseline, demonstrating strong potential for practical replayed deepfake detection.