Reinforcement learning improves spoken math problem solving accuracy
Voice of Reason: Reinforcement Learning for Spoken Math
Computation and Language
Summary
Speech models let people talk to computers more naturally, but they often struggle to solve math problems as well as text-based models. The authors took a speech model called GLM-4-Voice and trained it further with new methods, including reinforcement learning. This helped the model answer math questions more accurately than before, even without extra reasoning steps. Combining this with other techniques led to a new best performance for speech models on math tasks.
What this means in practice
- •For voice assistant developers: Improve the ability of voice assistants to understand and solve spoken math problems more accurately using reinforcement learning.$Commercial implications: This enables enhanced math-solving features in commercial voice assistants by improving speech-native model accuracy.
- •For telephony support teams: Deploy more accurate spoken math question answering in phone-based customer support systems for technical troubleshooting.
Authors
Timothée Weisselberger, Edouard Graves, Alexandre Défossez
Abstract
Speech language models enable richer spoken interactions between humans and machines than cascaded systems, allowing access to paralinguistic information and lower latency. However, their accuracy on mathematical reasoning benchmarks has lagged behind those of text models. Reinforcement learning (RL) with verifiable rewards has been instrumental in extending text models' capabilities for solving complex problems and limiting hallucinations. In this work, we explore applying RL to the GLM-4-Voice speech model (Zeng et al., 2024) to bridge the gap between textual and spoken mathematical problem solving. We first adapt the model to the domain using supervised fine-tuning on synthesized spoken question-answering data. We then show that, even without extra reasoning tokens, RL improves the accuracy on GSM8K beyond levels previously achieved for speech models only with supplementary reasoning traces. When combined with existing streaming reasoning techniques, we show further gains to 74.8% free-form accuracy. This establishes a new state-of-the-art for mathematical spoken abilities with speech-native models.