Video temporal grounding improved by group based reward verifier
Beyond Scalar IoU: Structured Verification from Rollout Groups for Video Temporal Grounding
Artificial Intelligence
Summary
Video temporal grounding is about finding the exact time intervals when something happens in a video. The authors found that previous methods scored each guess independently without considering how all guesses relate to each other. They developed a new approach, SUTURE, that looks at the entire group of guesses together to better judge their accuracy. This method leads to better performance on multiple video benchmarks and more accurate timing of events.
What this means in practice
- •For video platform developers: Improve tools that automatically find and label important moments in videos using group-aware reward verification.
- •For content moderation teams: Enhance detection of specific time sections in videos to support faster review of potentially sensitive content.
Authors
Youngjae Cho, Won Young Jhoo, Jongsuk Kim
Abstract
Reinforcement learning with verifiable rewards (RLVR) provides a natural framework for adapting pretrained models to video temporal grounding, where generated temporal intervals can be scored directly against ground truth intervals. Yet existing overlap verifiers typically score each rollout independently, leaving the joint structure of the rollout group unused. We introduce SUTURE, which conditions verification on the rollout group and exploits its structure at two complementary scales: disagreement across rollouts controls how strongly the target is reweighted, while coverage at each position determines where reward mass is redistributed. We show that the resulting verifier admits an exact decomposition into the standard IoU term and a covariance correction determined by the rollout group. A local gradient diagnostic finds a preference for responses covering relatively less supported target regions in the analyzed groups. Across five temporal grounding benchmarks, SUTURE improves grounding performance at every reported IoU threshold. Its trained policy also shows less video-start anchoring in reasoning traces: for later events, the first temporal mention more often overlaps the annotated target. Together, these results show that the joint structure of a rollout group can support a more informative temporal verifier.