Audio models improve themselves by judging their own reasoning
AudioRSI: Recursively Self-Improving Audio Reasoners
Sound
Summary
Audio reasoning models usually get better when guided by external rewards or judges. The authors show that an audio model can act as its own judge, scoring its own reasoning and improving itself in a cycle. This self-judging system helps the model avoid errors and keeps improving even on new, unseen audio tasks. Their approach outperforms other strong audio reasoning models on several test sets and also develops better logical and perception-aware reasoning.
What this means in practice
- •For audio software developers: Build audio understanding systems that improve their reasoning accuracy by self-evaluating without needing external feedback.
- •For voice assistant teams: Enhance voice assistants’ ability to reason about audio inputs more reliably in unfamiliar audio scenarios by recursive self-improvement.
Authors
Jiajun Fan, Roger Ren, Jingyuan Li, Rahul Pandey, Prashanth Gurunath Shivakumar, Ankur Gandhe, Yile Gu, Ge Liu, Ivan Bulyko
Abstract
Reinforcement learning has made Audio Large Language Models (Audio LLMs) better reasoners, but the reward that drives it still comes from outside the model: a final-answer check, a hand-crafted process score, or an external LLM judge that stays fixed while the policy improves. We show that an Audio LLM can supply this reward itself, and can do so recursively: each improved model becomes the critic that trains its successor, so the training signal improves along with the model. We introduce AudioRSI (Recursively Self-Improving Audio Reasoners), a reinforcement learning framework in which a single model plays two roles: as an actor it generates reasoning paths, and as a critic it compares them pairwise, turning its own judgments into a process-level reward. The critic is periodically replaced by the current policy, which closes the recursive loop. Because the critic hears the same recording as the actor, it can reject reasoning that cites acoustic evidence the audio does not contain, rather than reward reasoning that merely sounds plausible. The recursion is what sustains progress: under a frozen external judge, out-of-distribution accuracy plateaus during training, while a critic refreshed from the improving policy keeps it rising. Across four out-of-distribution benchmarks, all disjoint from the AVQA training distribution, AudioRSI outperforms strong baselines including Audio Flamingo 3 and Gemini 2.5 Pro on MMAU test-mini, ranks first on MMSU, achieves a clear lead on the challenging Multi-Audio split of MMAU-Pro, and leads the 7B open-source tier on MMAR among methods without external reward models. With no reward specifying how to reason, AudioRSI also develops structured reasoning behaviors (logical reasoning, perception-aware analysis, and reduced hallucination), corroborated by an external LLM-as-Judge evaluation and a keyword-based behavior analysis.