Consistency as Inductive Bias: Learning Cross-View Invariance for Robust Multimodal Reasoning

2026-06-29Computer Vision and Pattern Recognition

Computer Vision and Pattern Recognition
AI summary

The authors found that large multimodal language models (MLLMs) lack the ability to give consistent answers when shown different but equivalent views of the same input. They propose a method called ConsistRoll that encourages the model to give the same answer for these different views by giving rewards only when both views produce matching correct results. This approach improves learning without needing extra work like additional data or annotations. Their tests show it helps models reason better across various tasks.

Inductive BiasMultimodal Large Language ModelsCross-View ConsistencyReinforcement LearningReward FunctionData AugmentationCredit AssignmentGRPOAdvantage CollapseMultimodal Reasoning
Authors
Xin Zou, Haolin Deng, Yibo Yan, Shuliang Liu, Kening Zheng, Zhiwei Jin, Chen Chen, Haonan Lu, Xuming Hu
Abstract
Inductive biases steer learning toward generalizable solutions by encoding task structure. In this work, we identify a crucial missing bias in MLLMs: cross-view consistency, \textit{i.e.}, semantically invariant views of the same instance should lead to the same answer. Standard reinforcement learning with verifiable rewards (RLVR) objectives do not impose this constraint, but instead assign pointwise rewards to each visual input. Even with data augmentation (DA), transformed views are typically rewarded independently, providing little signal once within-view rewards saturate. We propose \textbf{ConsistRoll}, a simple but effective method that injects cross-view consistency into RLVR training by reusing the group-sampling mechanism of GRPO. Specifically, ConsistRoll places original and semantically invariant transformed views in the same generation group, and assigns a joint reward only when paired completions are both correct and consistent. In this way, ConsistRoll turns consistency into an online credit-assignment signal, \textbf{without extra generation overhead and annotations}. Theoretically, we show that cross-view consistency is a valid inductive bias, and ConsistRoll introduces a cross-view correction term absent from DA, penalizing view dependence and alleviating advantage collapse. Comprehensive benchmarks across math, general-purpose, hallucination domains confirm that ConsistRoll achieves robust improvements in multimodal reasoning.