Visual reasoning improved by exploring multiple image evidence branches
Latent Visual Contrastive Reasoning with Multi-Branch Exploration
Computer Vision and Pattern RecognitionComputation and Language
Summary
Sometimes AI models struggle to answer visual questions because they only follow one path when looking for clues in images. This paper presents a way to let the model consider several possible clues at once, and learn which clues are best using examples marked by humans. The model gets better at finding and comparing important parts of an image to answer complex questions. Tests show this method improves performance on demanding visual tasks compared to older methods.
What this means in practice
- •For ai application developers: Improve visual question answering systems by allowing multiple visual clue paths for better reasoning over complex images.
- •For computer vision engineers: Enhance image analysis systems by training models to align learned features with annotated regions for more accurate evidence detection.
Authors
Kegeng Tang, Jingbo Wang, Yangming Zhang, Huanshu Zhang, Shaogang Ren, Zihao Wang
Abstract
Multimodal large language models have made substantial progress in visual question answering, but complex visual reasoning often requires locating and comparing multiple pieces of question-relevant visual evidence. Existing methods that move intermediate reasoning into the visual latent space usually follow a single linear reasoning trajectory, which can be limiting when multiple visual hypotheses or competing evidence regions are relevant. We propose Latent Visual Contrastive Reasoning, a branch-aware latent reasoning framework that generates multiple question-conditioned latent visual branches as soft hypotheses over image tokens. To train these branches, our method uses bounding-box evidence annotations to build target evidence representations and introduces a soft contrastive branch-evidence alignment objective. This objective encourages evidence-relevant branches to align with annotated visual regions while contrasting them against negative evidence. We instantiate our method on Qwen2.5-VL backbones and train it with standard supervised fine-tuning. Experiments on perception-intensive visual reasoning benchmarks show that the proposed method achieves overall improvements over the corresponding backbone and single-path latent reasoning baselines at both 3B and 7B scales.