Video reasoning improved by transferring dense frame learning to sparse frame policies
Frame Differential On-Policy Self-Distillation for Video Reasoning
Computer Vision and Pattern Recognition
Summary
Video reasoning models need to understand important moments in videos, but looking at every frame is slow, and looking at only a few can miss details. The authors introduce a new training method called Frame Differential On-Policy Self-Distillation (FD-OPSD), which teaches the model to learn from detailed views of a video while using fewer frames during actual decision-making. This approach improves video understanding without extra cost when the model is used later. The authors tested this on several video reasoning tasks and showed better results than previous methods.
What this means in practice
- •For video analytics developers: Improve automated video understanding systems by training models that see fewer frames but still capture key events accurately.
- •For autonomous vehicle engineers: Enhance real-time scene analysis by reducing video input without losing critical temporal information during training.
Authors
Haiying He, Xin Zheng, Shaoli Hu, Shijun Xiao, Xuanhe Liu, Bing Li, Harry Yang
Abstract
Reinforcement learning (RL) has substantially improved the reasoning ability of multimodal language models through verifiable rewards and increasingly fine-grainedvisual or temporal credit assignment. In video reasoning, however, current RL methods typically train with a fixed sparse frame budget: increasing the number of frames makes autoregressive rollouts expensive, while too few frames may miss temporally localized events and fine-grained visual details. We present \textbf{Frame Differential On-Policy Self-Distillation (FD-OPSD)}, which transfers the useful evidence of dense frame observations to a sparse frame policy during RL training. FD-OPSD compares the policy's token level preferences for the same sampled response under sparse and dense views, and distills the resulting frame differential signal without an external teacher or dense autoregressive rollout. The method preserves sparse-frame rollouts and leaves inference unchanged. Across Qwen2.5-VL-7B and Qwen3-VL-4B on six video reasoning benchmarks, FD-OPSD yields higher overall average performance than the strongest corresponding GRPO, T-GRPO, or Video-KTR baselines across the 16, 32, and 64 frame evaluation settings. These results show that dense visual evidence can be transferred selectively during training through token level self-distillation while retaining sparse frame rollouts and unchanged inference.