Voice anonymization keeps speech clear and identity safe
The Content Leash: Mitigating Degenerate Solutions in Invertible Speaker Anonymization
SoundCryptography and Security
Summary
Protecting people’s voices when sharing speech data is important for privacy. This paper presents a method that hides who is speaking without making the speech hard to understand or ruining the original voice. The method also works well even against voice recognition tools it wasn’t specifically trained on. The authors show it can stop attackers from guessing the speaker while still letting authorized users restore the original voice clearly.
What this means in practice
- •For voice assistant developers: Enable voice assistants to share speech data protecting user identity while allowing restoration if authorized.
- •For call center operators: Process anonymized customer speech data to protect privacy without losing the ability to recover the original speaker’s voice when needed.
Authors
Nirjon Mobarak Talukder, Rafid Ahmed, Mohammad Ishrak Abedin, Abu Raihan Mostofa Kamal
Abstract
Voice anonymization aims to protect a speaker's identity while preserving the linguistic content of speech, enabling voice data to be shared and processed with reduced privacy risk. This work focuses on reversible voice anonymization, where the original voice can be recovered by an authorized user rather than being permanently discarded. However, existing approaches can achieve apparent speaker anonymization by degrading the speech itself, while their effectiveness may also depend on the speaker-verification representation used during training. This creates a need for methods that protect speaker identity without sacrificing speech intelligibility and that remain effective against unseen speaker verifiers. We introduce content preserving training constraints to improve the privacy-utility trade-off by preventing anonymization from being achieved primarily through speech degradation. We further introduce a verifier-generalization strategy that incorporates a stronger speaker-verification representation during training, improving anonymization robustness beyond the verifier used in the original training setup. The resulting approach maintains intelligible speech, strengthens anonymization, and preserves reliable voice restoration, with evaluation on unseen speakers and held-out speaker verification models. On 38 unseen speakers, the approach raises the attacker's equal error rate from a measured floor of 2.8% to 26.3%, and to 27.9% against a held-out verifier, with 11.8% word error rate and 94.9% speaker similarity on key-based restoration. These improvements advance reversible voice anonymization toward practical privacy-preserving speech applications, where anonymized speech can be safely processed or shared while the original voice remains recoverable under authorized conditions.