Video generation method improves reasoning using task specific guidance
VR-JEPA: Learning Contrastive-State Latent Guidance for Generation-based Video Reasoning
Computer Vision and Pattern Recognition
Summary
Making videos that show logical steps can be tricky because the changes between frames might not make sense or follow rules. The authors created VR-JEPA, a system that helps video generation better follow logical steps by learning from examples that succeed and fail. It trains model parts specialized for different tasks, so the generated videos are more physically accurate and logically consistent when used for reasoning. Tests show this method works better than earlier approaches.
What this means in practice
- •For computer vision engineers: Generate videos that visually demonstrate complex reasoning steps with fewer physical and logical errors.
- •For interactive media developers: Create more accurate and consistent video content that follows task-specific logical sequences for educational or training programs.
Authors
Zehua Ma, Kun Xiang, Yunshuang Nie, Quanlin Chen, Haoyuan Li, Xiuwei Chen, Jiang Ji, Haijun Wu, Zhenyu Xie, Michael Kampffmeyer, Hanhui Li, Xiaodan Liang
Abstract
Reasoning through video generation offers a promising path toward visual intelligence by modeling latent visual states and their dynamics. However, current video generation models often lack explicit guidance on how these states should evolve, leaving generated trajectories prone to physical and structural inconsistencies that undermine reasoning reliability. While the Video Joint-Embedding Predictive Architecture (V-JEPA) provides rich spatiotemporal priors learned through latent prediction, these general priors do not naturally adapt to the logical reasoning capabilities required for complex visual tasks. To bridge this gap, we propose VR-JEPA, a framework that aligns the V-JEPA predictor with task-specific reasoning logic through localized contrastive-state learning and uses its predicted latent trajectories to guide video generation for visual reasoning. Specifically, (i) we pair successful trajectories with generated alternatives under the same input conditions and use discrepancies in their V-JEPA representations to identify informative states and tokens for localized contrastive supervision. (ii) We further equip the V-JEPA predictor with skill-specific experts trained on anchor-task data, allowing the model to adaptively specialize its shared spatiotemporal priors across diverse cognitive domains. Together with skill-specific experts, this contrastive supervision enables VR-JEPA to predict latent trajectories that provide task-specific logical guidance for video generation. Comprehensive experiments on the large-scale VBVR-Pro-Bench dataset demonstrate that VR-JEPA achieves an $11.33\%$ relative improvement over the cutting-edge generation-based reasoning baseline, significantly mitigating physical artifacts and enhancing logical consistency.