X$^3$-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment
2026-07-23 • Machine Learning
Machine Learning
AI summaryⓘ
The authors address the challenge that audio-language models are not as good at logical reasoning as text-based models because there isn't enough audio reasoning data. They introduce X$^3$-OPD, a method where a strong text-based model teaches an audio-based model by guiding it through reasoning steps using related text. They also create a special dataset that includes speech, sounds, and conversations to help the model learn reasoning beyond just words, like sounds and tone. Their experiments show that this approach improves the audio model's reasoning without hurting its other skills.
audio-language modelslogical reasoningcross-modal distillationchain-of-thoughtspeech recognitionacoustic scenesparalinguistic cuestext-audio alignmentdomain shiftlarge language models
Authors
Dongjie Fu, Di Cao, Xize Cheng, Zihan Zhang, Wenxu Jia, Yifu Chen, Shengpeng Ji, Yu Zhang, Tao Jin
Abstract
While large audio-language models have achieved remarkable progress in auditory perception, they still lag behind text-based large language models in deep logical reasoning, primarily due to the scarcity of high-quality audio reasoning data. To bridge this gap, we propose X$^3$-OPD, a cross-modal on-policy distillation framework that transfers reasoning capabilities from a powerful text teacher to an audio-language student. During training, the student generates reasoning trajectories conditioned on its own acoustic perception, while the teacher provides token-level guidance using matched textual inputs and verified answers. We further construct a three-tier symmetric corpus covering textual reasoning rendered into speech, audio-event reasoning grounded in complex acoustic scenes, and spoken-dialogue reasoning involving paralinguistic cues. This design extends cross-modal distillation beyond textually recoverable content to reasoning grounded in non-linguistic events, prosody, and conversational context. Experiments on MMSU, MMAU, BIG Bench Audio, and MMAR demonstrate that X$^3$-OPD substantially improves audio-grounded reasoning and chain-of-thought quality while largely preserving the model's existing capabilities under domain shift.