Audio reasoning improves with reinforcement learning on sound tasks
Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks
SoundArtificial Intelligence
Summary
Understanding sounds and reasoning about them is harder than with text or images. The authors created Audio-R1, a way to train audio-language models using rewards from tasks where answers are known ahead, helping the model learn to reason better. This method boosts how well models understand sounds and helps with other audio tasks, even ones different from what the model was trained on. It shows that using reinforcement learning with audio tasks can improve audio comprehension and reasoning.
What this means in practice
- •For voice assistant developers: Enhance audio understanding and reasoning in virtual assistants to better interpret complex spoken inputs.
- •For smart home device engineers: Improve sound event detection and reasoning in devices through reinforcement learning with self-supervised audio tasks.
Authors
Jihoon Jeong, Mirco Ravanelli, Zhepei Wang, Cem Subakan
Abstract
Reasoning is increasingly important for large audio-language models (LALMs), yet remains relatively underexplored compared with text-only and vision-language models. We introduce Audio-R1, a reinforcement learning (RL) post-training framework that revisits audio self-supervised learning as a source of verifiable rewards for improving audio understanding and reasoning task performance. Audio-R1 uses answers automatically determined during the construction of self-supervised tasks to verify model responses and compute rewards. Experiments across multiple LALMs and diverse audio benchmarks show consistent performance gains, while ablations demonstrate the benefits of each self-supervised task and additional gains from joint training. While both supervised fine-tuning and Audio-R1 improve self-supervised task accuracy, Audio-R1 further improves performance on downstream tasks distinct from the training tasks. These results highlight the potential of RL post-training with self-supervised audio tasks to improve downstream performance and reasoning-trace quality.