Towards Robust Uncertainty-Aware Speaker Modeling
2026-07-06 • Sound
Sound
AI summaryⓘ
The authors work on improving how computers recognize who is speaking by making better use of 'uncertainty,' which tells how sure the computer is about its guess. They created a new method that pays attention to differences both between different people and the same person over time, helping the computer judge its confidence more accurately. They also designed a way to adjust the system when it faces new kinds of voice data, so its uncertainty estimates stay reliable. Tests showed their methods help the computer recognize speakers better, even when the voices come from unfamiliar sources.
Speaker embeddingsAcoustic featuresUncertainty estimationDomain adaptationIntra-speaker variabilityInter-speaker separabilityUncertainty calibrationSpeaker recognitionUncertainty softmax
Authors
Junjie Li, Yang Xiao, Kong Aik Lee
Abstract
Speaker embeddings aggregate frame-level acoustic features into compact representations for speaker recognition. Recent uncertainty-aware speaker modeling approaches further characterize the reliability of speaker embeddings by estimating their associated uncertainty. However, existing methods often suffer from inaccurate uncertainty estimation and uncertainty miscalibration under domain shifts. To address these challenges, we propose a robust uncertainty modeling framework from both estimation and adaptation perspectives. Specifically, we introduce an Inter- and Intra-Speaker-Aware Uncertainty Softmax that incorporates both inter-speaker separability and intra-speaker variability into uncertainty learning, enabling uncertainty estimates to better capture the reliability of speaker embeddings. Furthermore, we propose an Uncertainty-Calibrated Domain Adaptation (UCDA) framework to mitigate uncertainty miscalibration caused by domain mismatch. Extensive experiments on both in-domain and cross-domain benchmarks demonstrate that the proposed approach consistently improves uncertainty reliability and speaker recognition robustness.