Speaker voice can secretly trigger harmful AI commands
You Are the Trigger: Speaker-Activated Backdoors
Machine LearningCryptography and SecuritySound
Summary
Speech-based AI systems can be tricked by using a person’s natural voice as a secret key to activate harmful commands without the attacker needing to play any special audio at the time. The authors show a method to train models so that just hearing a specific speaker unlocks malicious actions or content changes in what the AI transcribes or does next. This attack works well on popular speech-to-text systems even with very short audio clips of the targeted speaker and can fool advanced AI assistants to carry out harmful instructions. The technique is invisible during normal use and works despite different recordings or noisy environments.
What this means in practice
- •For speech recognition engineers: Detect and defend against voice-activated backdoors that exploit speaker identity to hide malicious commands.
- •For ai security teams: Evaluate the risks of fine-tuning speech models on limited data that could covertly embed harmful behaviors triggered by specific speakers.
Authors
Nataša Krčo, Euodia Dodd, Peter Romov, Yves-Alexandre de Montjoye
Abstract
Speech models are increasingly used as an interface to AI systems, from transcription tools to voice agents. Existing audio backdoors require an attacker to introduce triggers at inference time. We study a different threat model in which the attacker compromises the model during fine-tuning but has no access to the victim's inference-time inputs. We introduce the Speaker-Activated Backdoor (SAB) setting, in which a target speaker's natural voice serves as the trigger. We develop a two-phase fine-tuning method that first makes speaker identity more accessible and then binds the target speaker to an attacker-chosen payload. Compared with a one-phase baseline with the same training budget, the two-phase method substantially improves selectivity and clean performance, and is especially important for content-dependent attacks. SAB achieves 90.2% and 97.0% backdoor success on Whisper-medium and Parakeet, with false-positive rates of 1.7% and 0%, respectively using as little as two minutes of real audio from the target speaker. We further demonstrate content-dependent semantic rewriting and downstream tool-use attacks, where malicious instructions are inserted into 96.9% of target-speaker transcripts. When these transcripts are passed to frontier LLM agents, GPT-4o and Claude (Opus, Sonnet, and Haiku), the injected instruction drives the agent to carry out the harmful action in up to 87.5% of cases. The attack generalizes to held-out recordings, multiple target speakers, and common acoustic degradations. These results establish speaker identity as a practical backdoor trigger that requires no inference-time attacker access and is invisible at the transcript layer.