Speaker diarization errors split to clarify pause ambiguity effects
Diarization Error Decomposition Under Pause Annotation Ambiguity
Sound
Summary
Speaker diarization means figuring out who spoke when in an audio recording. Sometimes, pauses between speakers are marked differently, which can make error rates look worse or hide real mistakes. The authors found that a common way to handle pauses loses important details about segments. They created a new method that breaks down errors into those caused by pause ambiguities and the core diarization mistakes, helping to better understand what went wrong.
What this means in practice
- •For speech technology developers: Improve diarization systems by pinpointing if errors are due to pauses or core misclassification during evaluation.
- •For automatic transcription services: Provide clearer feedback on speaker separation quality by separating errors caused by ambiguous pauses from true diarization faults.
Authors
Shota Horiguchi, Marc Delcroix, Naohiro Tawara, Alexis Plaquet
Abstract
Speaker diarization evaluation is sensitive to ambiguity in pause annotation, which can inflate diarization error rate (DER) or obscure genuine model errors. We show that morphological closing, which has been used for pause-tolerant diarization evaluation, discards segment-level distinctions. Instead, we propose an exact, overlap-aware decomposition of standard DER into a pause-attributable component, consisting of errors compatible with pause filling, and a residual core component that can serve as a proxy for intrinsic diarization errors. The decomposition leaves DER unchanged, while the pause-attributable and core components vary monotonically with the pause threshold and eventually saturate. Experiments spanning synthetic transformations, annotation mismatch, cross-domain evaluation, and tight-boundary diarization show that the decomposition reveals error sources not apparent from standard DER.