Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning
2026-07-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors discuss how current large multimodal language models struggle to combine visual reasoning smoothly because they rely on discrete tokens that may miss important details. They propose a new training method called Asymmetric Mutual Variational Learning (AMVL) to fix a mismatch between training and testing stages. This method uses two types of mathematical adjustments to make the model's reasoning paths more reliable and avoid cheating by using answers during training. Their tests show AMVL improves performance on complex reasoning tasks compared to other models and makes the internal reasoning more stable.
Multimodal Large Language ModelsContinuous Latent ReasoningTrain-Inference MismatchVariational LearningKL DivergencePosterior DistributionPrior DistributionBLINK BenchmarkLatent Space StabilityAnswer Leakage
Authors
Shijie Li, Yilin Gao, Siyuan Yang, Tieyuan Chen, Chaofan Gan, Zhihao He, Zicheng Zhao, Yuyu Guo, Weiyao Lin, Hang Yu
Abstract
Multimodal Large Language Models (MLLMs) are often constrained by a language-space bottleneck, forcing complex visual reasoning into discrete tokens which can lose perceptual nuance. A promising alternative is continuous latent reasoning, where the goal is to discover implicit reasoning pathways that bridge the multimodal query and the final answer. However, this introduces a severe train-inference mismatch: a training-time posterior, conditioned on the ground-truth answer, can exploit answer-dependent shortcuts. Standard variational training then forces the inference-time prior to mimic a posterior that has access to information unavailable at test time, leading to poor performance. To address this, we propose Asymmetric Mutual Variational Learning (AMVL), a framework that resolves this mismatch via a bidirectional calibration objective. A forward KL divergence trains the target-agnostic prior to match the posterior, while a novel reverse KL divergence simultaneously regularizes the posterior, preventing it from collapsing into inference-incompatible regions and mitigating this ``answer leakage''. We provide theoretical analysis formalizing this leakage as prior contamination and prove that our dual-KL objective reduces it. We instantiate AMVL in a latent-integrated MLLM and show that it consistently outperforms strong discrete and latent-reasoning baselines, improving the average score on the complex BLINK benchmark by +10.83 and achieving gains of up to +32.00 on individual reasoning tasks, with analyses confirming improved latent-space stability.