Speech genre affects training of turn-ending detectors in conversations
The Importance of Speech Genre for End-of-Turn Detector Training
Artificial Intelligence
Summary
Detecting when someone finishes speaking is important for smooth conversations, especially between humans and machines. This paper shows that training these detectors on real human conversations works better than training on one-turn exchanges with machines. The researchers found that models trained on conversations make fewer mistakes, especially with pauses and hesitations. This advantage only appears when testing on conversational speech, meaning most past work missed this benefit by focusing only on single-turn data.
What this means in practice
- •For voice assistant developers: Improve end-of-turn detection in assistants by training models on human conversation data to reduce false alarms from natural speech pauses.$Commercial implications: Enhances speech interaction quality for commercial voice-activated products by enabling smoother turn-taking.
- •For call center technology teams: Enhance speech analytics tools to better detect speaker turns in multi-turn customer-agent conversations by incorporating conversational training data.
Authors
Rini Sharon, Kadri Hacioglu, Jeena Prakash, Andreas Stolcke
Abstract
End-of-turn (EoT) detection has been studied mostly through features, models, and modalities, while the speech genre of the training data has drawn far less attention. We isolate this factor. Using one reproducible annotation pipeline, we build a matched pair of corpora, multi-turn human-human conversational and single-turn human-machine, and train the same streaming ConvLSTM EoT detector on each. Across four test sets the conversational model reaches up to +0.45 higher F1, driven by fewer false alarms on the mid-turn pauses and hesitations that are rare in single-turn corpora. This conversational-training effect is not specific to frame-level decoding. Under a matched complete-or-not test it also appears at the utterance level, across two encoders, but only on conversational test data. Testing only on single-turn speech, as most prior work does, therefore hides the benefit of conversational training data. Against open-source baselines, our 1.8M-parameter model attains the highest F1 on every test set.