Reasoning-Oriented Post-Training and Inference-Time LoRA Rescaling for Audio-Dependent Question Answering
2026-08-24 • Sound
Sound
AI summaryⓘ
The authors studied how well large audio-language models can answer questions based on sounds, focusing on a competition dataset. They tested a method called Low-Rank Adaptation (LoRA) to improve the models' reasoning about audio content and questions. Their experiments showed that LoRA helped one model (Qwen) but hurt another (MOSS-Audio), though tweaking the method during answering improved results for both. Overall, their approach ranked well in the challenge for models with fewer than 10 billion parameters.
Audio-Dependent Question AnsweringLarge Audio-Language ModelsLow-Rank AdaptationChain-of-ThoughtCross-Modal ReasoningPost-TrainingInference-Time RescalingDCASE ChallengeQwen2.5-OmniMOSS-Audio-8B-Thinking
Authors
Weiteng Hu, Yin Cao, Jun Yang
Abstract
Audio-Dependent Question Answering (ADQA) requires Large Audio-Language Models (LALMs) to answer questions whose correct answers depend on the given audio content. Successful ADQA requires accurate audio perception, identification of question-relevant evidence, and cross-modal reasoning. Using the official ADQA dataset of DCASE 2026 Task 5, we investigate reasoning-oriented post-training with Low-Rank Adaptation (LoRA) and inference-time LoRA rescaling for both Qwen2.5-Omni and MOSS-Audio-8B-Thinking. We introduce a structured Chain-of-Thought (CoT) framework that decomposes the reasoning process into question analysis, question type, audio evidence, and reasoning. We then analyze how task-specific LoRA adaptation affects the two backbones and further explore inference-time rescaling of trained LoRA adapters. Experiments on the development set reveal markedly backbone-dependent behavior: post-training improves the Qwen-based systems but substantially degrades MOSS-Audio under our supervised fine-tuning configuration. Moderate LoRA rescaling further improves the best Qwen system's top-1 accuracy from 58.93% to 61.05% and partially restores the performance of the fine-tuned MOSS-Audio models, while the best MOSS-Audio system achieves 67.70% top-1 accuracy. Our submitted systems ranked third overall and second among lightweight systems under 10B parameters in the challenge.