Framegrpo improves video reasoning by using multiple frames and smarter rewards
FrameGRPO: Incentivizing Video Reasoning via Multi-Frame Rollouts and Temporal Credit
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Video reasoning helps computers understand and answer questions about videos. The authors found that current methods use the same video frames in many tests, which can limit learning. They propose FrameGRPO, which looks at different sets of video frames during training and gives better rewards when shorter views solve problems correctly. This approach helps computers learn to reason about videos more effectively without adding complex parts to the model.
What this means in practice
- •For video analytics developers: Improve video understanding models by training them using diverse frame samples to enhance reasoning accuracy.
- •For autonomous vehicle engineers: Enhance decision-making systems by integrating video reasoning that better handles temporal variation and uncertain visual inputs.
Authors
Jiaxuan Sun, Biao Wang, Shuai Li
Abstract
Reinforcement learning with verifiable rewards, particularly Group Relative Policy Optimization (GRPO), has improved video reasoning in multimodal large language models. However, standard video GRPO varies only linguistic trajectories while conditioning every rollout in a group on the same sampled frames, which can produce homogeneous rewards and weaken group-relative learning when the shared visual evidence is insufficient or distracting. We propose \textbf{FrameGRPO}, a simple and effective reinforcement-learning framework that requires neither auxiliary networks nor changes to the model architecture. Specifically, FrameGRPO combines two lightweight components: (1) \textbf{Multi-Frame Rollouts}, which use only uniform temporal subsampling of a single maximal-frame decode to construct aligned views that diversify visual evidence and reasoning trajectories; and (2) \textbf{Gated Temporal Efficiency Credit}, which selectively reinforces a correct lower-ratio trajectory when all rollouts in a fixed full-ratio reference subset fail, without imposing an unconditional preference for fewer frames. Extensive experiments across diverse video understanding and reasoning benchmarks demonstrate robust overall improvements over standard GRPO, highlighting temporal-view diversification as an effective mechanism for exploration and credit assignment in video MLLMs.