Aural speeds up speech model reasoning by predicting multiple steps ahead
AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models
Computation and LanguageMachine LearningSound
Summary
Getting speech language models to think carefully and answer quickly at the same time is hard. The authors show a way to speed up the thinking part by imagining several possible thought steps all at once, instead of one by one. They trained their system on a large set of bilingual speech data with explanations, and then improved it using trial and error to decide how much thinking to do for each question. Their approach makes the model answer almost as well as slower methods, but much faster, especially on harder questions that need more reasoning.
What this means in practice
- •For voice assistant developers: Reduce response time of speech models when processing complex user requests by predicting reasoning steps in parallel.
- •For call center technology teams: Improve emotion recognition and empathetic dialogue systems with faster and adaptive reasoning over spoken inputs.
Authors
Yuxiang Wang, Kunyu Feng, Yuancheng Wang, Zihang Liu, Shengbo Cai, Qinke Ni, Wan Lin, Tao Feng, Yingda shen, Ming-Hao Hsu, Zhixian Zhao, Liqiang Zhang, Teddy Sun, Steve Yves, Zhizheng Wu
Abstract
Model intelligence and fast response jointly shape the quality of interaction with speech language models, yet remain difficult to achieve together. Explicit chain-of-thought (CoT) improves reasoning and audio understanding, but generating intermediate reasoning tokens delays responses. Describing fine-grained acoustic cues further lengthens CoT and increases latency. Latent reasoning can reduce this overhead, yet existing methods often trail CoT and remain limited by single-path supervision and reasoning budgets that do not adapt to problem difficulty. We introduce AURAL, which models a distribution over multiple plausible reasoning continuations in latent space and jointly predicts chunks of future states to reduce sequential forward passes and reasoning latency. To provide initial supervision for latent reasoning, we construct AuralReason-683K: 683K bilingual speech utterances (about 1,000 hours) with concise CoT for emotion recognition, empathetic dialogue, and general reasoning. AURAL-RL then explores beyond these traces, rewarding concise reasoning that yields high-quality answers and adapting reasoning effort to each problem. Across two backbones, AURAL-RL achieves performance comparable to CoT-RL, with larger gains over the respective supervised checkpoints on most metrics. Analysis further shows that harder questions elicit more latent reasoning steps. On Qwen2.5-Omni, it reduces time to the first answer token by 11.8x, from 1.22 to 0.10 s, versus 0.05 s for direct answering.