Speaker diarization improved for live and recorded audio
S2SND-v2: Unified Online and Offline Speaker Diarization with Speaker-Adapted w2v-BERT 2.0
Sound
Summary
Speaker diarization is the process of telling who spoke when in an audio recording or live conversation. This paper presents improvements to a system called S2SND-v2 that work both for online (live) and offline (recorded) scenarios. They enhance the system’s ability to understand speech features and handle overlapping voices more effectively. As a result, their method achieves competitive error rates on multiple test sets, meaning it can identify speakers more accurately in challenging conditions.
What this means in practice
- •For call center operators: Improve real-time speaker tracking in multi-person customer service calls to enhance monitoring and analytics accuracy.
- •For meeting transcription teams: Generate more accurate speaker labels in offline transcriptions of meetings where multiple participants speak and sometimes overlap.
Authors
Jincheng Wang, Li Li, Weixin Zhu, Yannan Wang, Tao Yu, Juan Liu, Ming Li
Abstract
This paper presents S2SND-v2, a unified online and offline speaker diarization framework built on S2SND that improves acoustic feature extraction and the use of the resulting representations. For acoustic feature extraction, we introduce a speaker-adapted w2v-BERT 2.0 front end and compare full and pruned construction routes. To better use these features, overlap-aware activity-query masking improves speaker embedding extraction under overlap, while late-speaker training aligns unknown-speaker training with online inference conditions. Combining these improvements, the pruned system achieves competitive average diarization error rates of 14.52% online and 12.03% offline across eight test sets.