Audio language models struggle with child stuttering speech understanding
Reasoning Beyond Transcription: Audio Language Models on Child Stuttering Speech
Computation and Language
Summary
Understanding speech from children who stutter is hard because their speech sounds, rhythms, and word patterns are different and often interrupted with repetitions. The authors studied if special audio language models can still make sense of what these children say, even when adults are also speaking. They tested if the models could summarize or judge sentences focused on the children, keeping important stuttering details. The results showed the models do understand the main ideas but have trouble when the stuttering or mixed voices increase.
What this means in practice
- •For speech therapy software developers: Improve tools that analyze child stuttering by using audio models that keep disfluency details and focus on child speech within mixed conversations.
- •For voice assistant designers: Enhance recognition systems to better understand children’s stuttered speech in multi-speaker environments without needing speaker separation.
Authors
Chibuzor Okocha, Christan Grant, Zoey Liu
Abstract
Child speech differs from adult speech in acoustics, prosody, and linguistic structures. Speech disfluencies (such as repetitions) further challenge automatic understanding. While Audio Language Models (ALMs) show strong semantic reasoning from speech audio, their ability to reason about disfluent child speech in mixed-speaker settings remains unexplored. We investigate this through two tasks: child-focused semantic summarization and speech entailment. Experiments use recordings of children who stutter in mixed speaker interviews without explicit speaker separation. Models are instruction-guided to focus on the child, preserve clinically relevant disfluencies, and avoid adult-speech leakage. Evaluation combines LLM-based judges and reference-based metrics, anchored by transcript-oracle baselines to isolate errors. Results show that while ALMs extract high-level meaning from stuttered speech, reasoning degrades significantly with increased