Acoustic and content attacks impact multilingual voice anonymization effectiveness
Exploiting Acoustic and Content-Oriented Speaker Verification Attacks Against Multilingual Voice Anonymization
SoundArtificial Intelligence
Summary
Voice anonymization tries to hide who is speaking while keeping the message understandable. The authors studied how attacks on voice anonymization work across different languages. They found that attacks focusing on sound features generally work better, but when the voice still sounds clear and content is preserved, attacks focusing on speech content get closer in success. They also created a new multilingual voice dataset that helps attackers across languages do better. This shows that protecting voice privacy requires understanding both the sound and the speech meaning.
What this means in practice
- •For voice security teams: Improve evaluation of voice anonymization systems by considering attacker models that use both sound and speech content across multiple languages.
- •For multilingual voice assistant developers: Enhance voice anonymization tools to better protect user identity while maintaining language utility, improving privacy for multilingual users.
Authors
Ridwan Arefeen, Ze Li, Rong Tong, Ming Li, Xiaoxiao Miao
Abstract
Attacker ASV systems for voice anonymization have been studied primarily in English, leaving their behavior in multilingual settings largely unexplored. Conventional ASV has shown that both acoustic and contextual information are important for multilingual speaker verification. Inspired by this, we investigate whether the same holds for attacker ASV on anonymized speech. We evaluate both acoustic- and content-oriented attackers on multilingual anonymized speech and construct a multilingual voice-converted dataset to improve cross-lingual generalization. Our results show that attacker effectiveness depends on the linguistic utility of the anonymized speech. Overall, acoustic-oriented attackers achieve better performance. However, when linguistic information is well preserved, the performance gap between content- and acoustic-oriented attackers narrows compared with conditions involving stronger speech distortion. The multilingual voice-converted dataset further improves performance and partially reduces the cross-lingual gap. These findings highlight the need for more comprehensive attacker modeling and evaluation protocols that consider both privacy and utility, rather than relying on a attacker strategy\footnote{Full code and pretrained models and MultiVC Dataset link are available at: https://github.com/monkeyDarefeen/DAST