MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning
2026-04-09 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors propose MedVR, a new method that helps medical AI models use pictures more clearly to make decisions without needing extra labels. They designed two main strategies: one guides the model to focus on uncertain parts of the image, and the other uses agreement in repeated tries to teach itself. This approach improves how well the models answer medical questions about images, doing better than previous methods. By making the AI reason directly from images, the authors aim to make it safer and more reliable for real medical use.
Medical Vision-Language ModelsReinforcement LearningEntropy-guided Visual RegroundingConsensus-based Credit AssignmentVisual ReasoningVisual Question AnsweringModel UncertaintyPseudo-supervisionVisual HallucinationClinical AI
Authors
Zheng Jiang, Heng Guo, Chengyu Fang, Changchen Xiao, Xinyang Hu, Lifeng Sun, Minfeng Xu
Abstract
Medical Vision-Language Models (VLMs) hold immense promise for complex clinical tasks, but their reasoning capabilities are often constrained by text-only paradigms that fail to ground inferences in visual evidence. This limitation not only curtails performance on tasks requiring fine-grained visual analysis but also introduces risks of visual hallucination in safety-critical applications. Thus, we introduce MedVR, a novel reinforcement learning framework that enables annotation-free visual reasoning for medical VLMs. Its core innovation lies in two synergistic mechanisms: Entropy-guided Visual Regrounding (EVR) uses model uncertainty to direct exploration, while Consensus-based Credit Assignment (CCA) distills pseudo-supervision from rollout agreement. Without any human annotations for intermediate steps, MedVR achieves state-of-the-art performance on diverse public medical VQA benchmarks, significantly outperforming existing models. By learning to reason directly with visual evidence, MedVR promotes the robustness and transparency essential for accelerating the clinical deployment of medical AI.