Video model improves understanding by adaptive evidence gathering
VLX-VR: An Agentic-Aware Video Reasoning Model
Computation and LanguageComputer Vision and Pattern Recognition
Summary
Understanding videos needs looking at pictures, sounds, words, and time clues together. Many video systems look at fixed parts once, missing important details when things are unclear. The authors created VLX-VR, a system that decides what information to check next and remembers what it learned, improving how it understands videos. It uses trial and error learning and works well on a video reasoning test with good accuracy and consistent reasoning.
What this means in practice
- •For video analytics teams: Build video analysis tools that adaptively choose relevant evidence to improve interpretation of complex scenes.
- •For robotics programmers: Create robot vision modules that handle ambiguous or incomplete video inputs by iterative evidence gathering and memory use.
Authors
Sheng Li, Peng Liu, Qianqian Zhang, Tiancheng Zhao
Abstract
Real-world video understanding requires integrating visual, audio, textual, and temporal evidence distributed across a video. Yet many pipelines use a fixed video context and single-pass inference, limiting adaptive evidence acquisition when observations are incomplete, ambiguous, or conflicting. We present VLX-VR, an agentic-aware video reasoning model trained within a video reasoning framework defined by a Think--Memory--Observation loop. At each step, VLX-VR determines the needed evidence, invokes read_memory or write_memory, incorporates the returned Observation, and decides whether to continue or produce the task output. We train VLX-VR with multimodal data, including videos and agent trajectories, using reinforcement learning to learn evidence acquisition, memory use, and termination. On MINERVA, VLX-VR achieves state-of-the-art performance among the models included in our comparison, with 78.79% accuracy. Under the original three duration groups, its accuracies are 76.70%, 78.73%, and 80.92%, with a cross-duration accuracy variance of 2.97~$\mathrm{pp}^2$. On correctly answered samples, 96.20% of VLX-VR's reasoning traces are consistent with the MINERVA reference reasoning traces and the evidence described by them, while approximately 75.80% of all evaluated samples satisfy both answer correctness and this evidence-grounded trace criterion. These results show strong performance and broadly stable behavior across durations, while counting, state changes, causal reasoning, and spatial perception remain challenging.