Self supervised models improve speaker verification across languages
Cross-Lingual Speaker Verification with Self-Supervised Pre-Trained Models
Sound
Summary
Identifying if a voice belongs to a specific person can get harder when people speak different languages. The authors found that by using self-supervised models trained on lots of varied speech, they can separate a person's voice traits from language traits. This helps create speaker recognition systems that work well even when the language changes. They tested their approach on a public benchmark and found it reduced mistakes in speaker verification.
What this means in practice
- •For voice biometric system developers: Build speaker verification systems that maintain accuracy despite users speaking different languages.$Commercial implications: Enables creation of commercially viable cross-language voice authentication products for global markets requiring reliable identity recognition.
- •For call center technology teams: Improve identification of callers in multilingual environments to enhance customer authentication and security.
Authors
Jinghan Peng, Yu Zheng, Weiqiang Wang, Jian Liu
Abstract
Speaker verification (SV) performance degrades under language mismatch due to the entanglement of speaker identity with language-specific acoustic cues. To address this problem, we leverage large-scale self-supervised pre-trained models (PTMs) to learn language-agnostic speaker representations. We utilize PTMs as robust front-end feature extractors, capitalizing on their rich acoustic and linguistic knowledge acquired from vast, diverse audio data. These generalized features are then used to train a downstream speaker embedding network, effectively disentangling speaker identity from language-specific characteristics. We validate our approach on the TidyVoice2026 benchmark, which benchmarks SV under language mismatch. Our proposed system (team T02) achieves equal error rates (EERs) of 2.21% on tv26_eval-A and 2.99% on tv26_eval-U.