Visual agents read ultra-high-res photos accurately by comparing image versions

ViRA: Toward Ultra-High-Resolution Visual Agents via Cross-Input Relative Policy Optimization

Computer Vision and Pattern RecognitionArtificial Intelligence

Summary

Reading tiny details in huge photos is hard for AI agents because they often guess based on common sense instead of actual evidence. The authors designed a new training method called CrossRPO that makes agents compare different versions of an image with slight changes to ensure they really look at the evidence. They created special training and testing sets with ultra-high-resolution photos to teach and evaluate these agents. Their trained agents performed better than previous models at answering questions correctly by truly reading the image details.

What this means in practice

  • For image analysis teams: Improve accuracy of photo interpretation systems by training agents to read fine details in ultra-high-resolution images using contrast-based reinforcement learning.
  • For ai development teams: Build visual reasoning agents that better validate their answers by learning to distinguish subtle image changes, enhancing trustworthiness in vision-based AI tasks.

Authors

Huawei Lin, Jinting Liu, Tongyan Wang, Xuyang Chen, Ruihua Shi, Zhichao Xu, Jiaxin Qin, Guoshenghui Zhao, Baijian Yang, Yingjie Victor Chen, Tan Yu, Weijie Zhao

Abstract

Visual agents answer questions about ultra-high-resolution photos by zooming into the relevant regions. However, existing agents often answer from common sense instead of reading the evidence. Reinforcement learning (RL) on the final answer cannot correct this, because a guess earns the same reward as reading. We propose Cross-Input Relative Policy Optimization (CrossRPO), which trains agents on contrast sets. A contrast set asks one question about several versions of an image that differ only in a small region with the evidence and have different answers, so only reading the evidence solves the whole set. CrossRPO rewards solving the whole set, and we prove that it does not reinforce a set answered from common sense. We build contrast sets for training (ViRA-Train) and evaluation (ViRA-Bench) by editing one small object in images of 21 to 120 megapixels, and we photograph further sets with real changes (ViRA-Bench-Real) to exclude editing traces. We train two Visual Reasoning Agents, ViRA-9B and ViRA-27B, with CrossRPO on only about 3,500 items of ViRA-Train each. CrossRPO achieves 63.3% set accuracy on ViRA-Bench at the maximum resolution, against 59.3% for GRPO. ViRA-9B is the most accurate open model of its size on all benchmarks, and ViRA-27B outperforms Claude Haiku 5.5 and GPT-6-Luna, both at their highest thinking level, on most benchmarks.