LoopSLM improves spoken dialogue empathy with faster acoustic reasoning
Thinking in Depth, Speaking Directly: Recurrent Latent Reasoning for Paralinguistically Grounded Spoken Dialogue
Computation and LanguageArtificial Intelligence
Summary
Spoken dialogue systems need to understand not just the words people say but also how they say them, like tone and emotion. The authors found that simply making acoustic cues explicit isn't enough for the system to plan good responses. They created LoopSLM, a new method that repeatedly refines its understanding using sound details to think more deeply before replying. This approach reduces slowdowns, boosts accuracy in understanding, and produces better, more empathetic replies.
What this means in practice
- •For voice assistant developers: Create voice assistants that better understand emotions and tones to generate empathetic, context-aware replies with lower delay.
- •For customer service chatbot teams: Improve customer chatbots’ ability to detect and respond to emotional cues in spoken conversations, enhancing user satisfaction.
Authors
Shengbo Cai, Yuxiang Wang, Jingran Xie, Zhisheng Zhang, Shun Lei, Di Cao, Teddy Sun, Zhiyong Wu
Abstract
Empathetic spoken dialogue requires models to use both what is said and how it is said to decide how to respond. Explicit CoT can improve paralinguistic perception and make acoustic cues more explicit in replies, yet does not ensure their effective use in response planning. We call this mismatch the perception-reasoning gap. In addition, CoT may not fully capture acoustic cues in words, and generating it adds inference latency. To address these limitations, we introduce LoopSLM, which builds on looped Transformers for latent reasoning, reusing a decoder block to refine hidden states with acoustic grounding at every pass. Its two-stage training further narrows the perception-reasoning gap by separating learning to reason from learning to respond, enabling direct inference without CoT. On EchoMind, LoopSLM improves paralinguistic understanding, reasoning, and reply quality over Qwen2.5-Omni-7B. Against the CoT-SFT baseline, LoopSLM gains over 20 points in reasoning accuracy while generating 64.5% fewer tokens at half the latency. It also outperforms Qwen3-Omni-Thinking on most empathetic reply metrics with 34x lower latency. Despite training only on dialogue data, LoopSLM improves accuracy on general audio benchmarks.