CAVE: Competence-Aware Visual Boundary Evidence Alignment for Video Temporal Grounding
2026-08-03 • Computation and Language
Computation and LanguageComputer Vision and Pattern Recognition
AI summaryⓘ
The authors studied how large vision-and-language models predict exact times in videos when describing events, finding that current methods often mismatch visual clues and their predicted time points. To fix this, they created a method called CAVE that uses special tokens to focus on visual details at the start and end of events, helping the model better line up visual evidence with the correct timestamps. They also designed a smart way to adjust guidance depending on how well the model is doing, preventing too much restriction as predictions improve. Their tests on common video datasets showed that this approach helps improve video event timing accuracy.
Video Temporal GroundingLarge Vision-Language ModelsReinforcement LearningTimestamp PredictionVisual Boundary EvidenceAttention MechanismEvidence-Timestamp AlignmentSupervised Warm-upLocalization OptimizationPerformance-aware Gating
Authors
Wei Jia, Zhicong Lu, Yu Chen, Xiang Wang, Shuai Li, Wenqian Lv, Jiayue Cao, Huaxing liu
Abstract
Large vision-language models (LVLMs) have achieved substantial performance gains in Video Temporal Grounding (VTG) through reinforcement learning (RL). However, existing methods primarily rely on outcome correctness rewards that evaluate only the final predicted intervals, leaving boundary-related visual evidence and its correspondence with timestamp predictions insufficiently constrained. In this paper, we delve into timestamp prediction and its underlying boundary-level visual evidence, showing prevalent misalignment between visual evidence and predicted timestamps across widely used benchmarks. To address this issue, we propose Competence-Aware Visual Boundary Evidence Alignment (CAVE), which augments localization optimization with boundary-specific visual evidence rewards to mitigate evidence-timestamp misalignment. Specifically, to explicitly represent the boundary-specific visual evidence, CAVE introduces boundary-specific evidence tokens and initializes their structured generation and distinct boundary semantics through a lightweight supervised warm-up. During RL, the visual boundary evidence alignment reward reinforces the visual attention of special evidence tokens within the ground-truth boundaries, thereby promoting alignment between visual evidence and temporal boundaries. Moreover, performance-aware gating for evidence supervision is designed to adaptively retain evidence guidance for poorly localized groups while reducing it once localization becomes sufficiently accurate to avoid over-constraining fine-grained boundary refinement. Extensive experiments on several public VTG benchmarks demonstrate the effectiveness of our method.