The Role of Disfluencies in Speech Translation
2026-08-03 • Computation and Language
Computation and Language
AI summaryⓘ
The authors found that current speech translation systems usually remove speech disfluencies like false starts and filled pauses, but these disfluencies carry important meaning that gets lost. They created Uh-Mazing, a new test set with translations that keep track of these disfluencies across eight languages. Their experiments showed that missing false starts and self-corrections causes most translation errors, and the models often just leave these parts out instead of mistranslating them. They also demonstrated a way to fix this problem during translation without needing to retrain the model.
speech translationdisfluenciesfilled pausesfalse startsself-repairsSwitchboard corpusSpeechLLMsinference-time decodingbenchmark dataset
Authors
Maike Züfle, Maria Teleki, Fabian Retkowski, Vilém Zouhar, Oliver Grabner, Alexander Waibel, James Caverlee, Jan Niehues
Abstract
Current speech translation systems, including SpeechLLMs, are trained on cleaned text and tend to strip disfluencies like filled pauses and false starts rather than translate them. We show this comes at a cost: disfluencies carry meaning that gets lost when speech is cleaned up. To study this systematically, we introduce Uh-Mazing, a benchmark of human-translated, disfluency-annotated Switchboard speech covering English into eight target languages. Across these languages and several architectures, we find that false starts and self-repairs, not filled pauses or discourse markers, drive most of the translation-quality loss, and that models which fail to preserve a disfluency tend to omit it rather than mistranslate it. We show inference-time decoding can mitigate this without retraining, and release the benchmark and code.