MIRA improves multimodal reasoning robustness against deceptive visuals
MIRA: Reinforcing Multimodal Reasoning via Deceptive Contextual Augmentation
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Multimodal AI systems that combine images and text sometimes make mistakes because they get tricked by confusing or slightly wrong visual details. The authors introduce MIRA, a new training method that makes these systems better at spotting and correcting errors when the visuals don’t match the logic they’re using. MIRA works by giving the AI tricky visual clues during learning so it learns to reflect and fix inconsistencies on its own without needing extra human help. Tested on different benchmarks, MIRA consistently improved how well the AI handles both understanding and reasoning tasks involving images and text.
What this means in practice
- •For multimodal ai developers: Train AI models that better detect and fix errors caused by confusing visual inputs in image-and-text reasoning tasks.
- •For computer vision engineers: Improve robustness of vision systems by incorporating deceptive scenarios during training to handle unexpected or misleading visual cues.
Authors
Zhihan Yin, Jianxin Liang, Yifeng Yao, Nonghai Zhang, and Bingyue Peng, Chunguang Qie, Huishuai Zhang, Dongyan Zhao
Abstract
While Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced multimodal reasoning, existing frameworks suffer from a critical "perception-reasoning gap." Due to an over-reliance on seemingly relevant visual tokens, even minor perceptual perturbations can propagate through the reasoning process, leading to compounding hallucinations. To address this issue, we propose MIRA, a training framework that improves robustness to erroneous visual contexts while enhancing logical reasoning ability. By injecting filtered, deceptive visual contexts during training, we construct challenging "hard traps" that stress-test the model's robustness to misleading information. Furthermore, we introduce a group-wise reflection-triggering mechanism that enables the model to autonomously detect and rectify inconsistencies between visual cues and logical constraints, which is ultimately internalized as an intrinsic policy behavior. Importantly, MIRA does not rely on human annotations. Experimental results across multiple reasoning and perception benchmarks show that MIRA yields an average 7% improvement on in-domain benchmarks, with consistent improvements on vision-centric benchmarks. These findings highlight that active error recovery is essential for robust multimodal reasoning. Comprehensive ablation studies and analyses further provide insights into why MIRA is effective.