LiveAnon improves speaker privacy and emotion in live speech anonymization
LiveAnon: Improving Privacy and Utility in Streaming Speaker Anonymization with Codec Language Models
Machine LearningSound
Summary
People want to keep their voice identity private but still show what they say and how they say it, like their emotions. Existing live systems often lose emotion or need special training for it. This team made LiveAnon, a system that changes voice identity in real-time while keeping emotions, by using a special language model and new decoding method to be fast. Their system proved to keep voices understandable and emotional while protecting privacy well.
What this means in practice
- •For voice assistant developers: Enable real-time anonymized voice input that keeps emotional cues for better user experiences while protecting speaker identity.$Commercial implications: Supports building privacy-preserving voice assistants that maintain natural emotion, improving user trust and engagement.
- •For call center technology teams: Implement live voice anonymization in calls to protect customer privacy without losing emotional content important for service quality.
Authors
Yejin Kim, Kyungsu Kim, Seungu Han, Kyogu Lee
Abstract
Speaker anonymization (SA) aims to conceal speaker identity while preserving linguistic content and emotion, but existing streaming systems often lose emotion information through discrete content bottlenecks or require emotion-specific modeling. We present LiveAnon, a voice-conversion-based streaming SA system that directly conditions an autoregressive codec language model on hidden features from a pretrained streaming ASR encoder, preserving emotion without emotion-specific supervision. To reduce codec-generation latency, we introduce capped RVQ decoding, which preserves dependencies among coarse codebooks while predicting the remaining codebooks in parallel. Under VPC24, LiveAnon improves intelligibility and emotion preservation over fixed-target Stream-Voice-Anon (SVA), a strong prior streaming SA system, matches emotion-prompted SVA in emotion preservation, and achieves the strongest semi-informed privacy among streaming systems. Under VPC26, it ranks second across all metrics among the evaluated anonymization systems, demonstrating a strong privacy-utility balance.