Using Prosody to Predict Syntactic Structure
2026-08-31 • Computation and Language
Computation and LanguageArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors studied how the way we speak (prosody), like how long words are and the pauses between them, relates to sentence structure (syntax). They used a method based on information theory to measure how much these speech features tell us about syntax, analyzing both audiobook reading and casual conversations in English. Their approach works with different types of speech data and can look at specific prosodic features separately. They found that prosody does provide useful clues about sentence structure, especially in spontaneous speech where it helped reduce uncertainty about syntax by about 10%. This supports some existing ideas about how speech patterns connect to grammar.
prosodysyntaxmutual informationinformation theoryspeech-text corporamultimodal language modelsword durationinter-word pausessyntax-prosody interfacespontaneous conversation
Authors
Junghyun Min, Alex Warstadt, Tamar I. Regev, Tiago Pimentel, Ethan Gotlieb Wilcox
Abstract
While it is well-established that prosody carries crucial cues for syntactic structure, the degree and nature of correspondence between these two domains remains contested. We investigate the syntax-prosody interface through an information-theoretic lens, quantifying the interaction between prosodic features and syntactic representations as their mutual information. We provide a general-purpose framework for estimating this quantity over large speech-text corpora using multimodal language models. Our framework is structure-agnostic and modular, insofar as it can be used to measure the contributions of individual prosodic features or components of structure. We evaluate the syntax-prosody relationship for two features (word duration and inter-word pauses) across two domains--read audiobooks and spontaneous conversations--both in English. Our results demonstrate that prosody contains measurable syntactic information, with prosodic features reducing syntactic uncertainty in spontaneous conversations by up to 10.2%. Our findings offer new empirical support for several theoretical accounts of the syntax-prosody interface.