Audio language model adapted to detect speech spoofing attacks
Spooftral: Can Voxtral Audio-Language Model Detect Speech Spoofing?
Computation and Language
Summary
Detecting fake or manipulated speech is important for security systems. The authors studied how an existing model that understands both speech and text handles tricked audio meant to fool such systems. They found that without extra training, the model focused on the meaning of words rather than subtle audio clues that reveal fakes, making it less effective. After a small additional training step, they improved the model’s ability to spot fake speech with a good accuracy score.
What this means in practice
- •For security system developers: Integrate adapted audio-language models to improve detection of spoofed speech in voice authentication systems.
- •For voice assistant developers: Enhance voice assistants to better recognize and reject manipulated or fake user commands generated by spoofed speech.
Authors
Avishai Weizman, Yehuda Ben-Shimol, Itshak Lapidot
Abstract
Self-supervised learning (SSL) countermeasures (CMs) have shown strong performance in recent years. However, they often show degraded performance while facing unseen spoofing attacks and mismatched conditions. This study examines the Voxtral audio-language model (ALM) framework for spoofing detection, as a step toward combining CM capabilities within the ALM framework. We analyze how Voxtral captures spoofing cues through audio-text processing and propose an instruction-guided approach that uses label-sequence likelihoods to evaluate bonafide and spoofed speech. Experiments on the ASVspoof databases show that without task-specific adaptation, the LLM layers emphasize semantic representations, reducing the separability of spoof-discriminative acoustic cues compared to the Whisper-based audio encoder. Consequently, spoofing-related information becomes less separable after language-model processing. We also applied lightweight adaptation using weight-decomposed low-rank adaptation (DoRA) to the Voxtral model and propose the Spooftral model, achieving an equal error rate (EER) of 4.25% on the ASVspoof5 evaluation set.