Video retrieval improves by verifying evidence before final selection
Concentrate After Imagination: Text-Conditioned Evidence Grounding for Partially Relevant Video Retrieval
Computer Vision and Pattern Recognition
Summary
When people search videos using phrases that describe only small parts of long clips, finding the right moments is tricky. The authors found that current methods sometimes pick the wrong parts because they rely only on the strongest matching snippet without double-checking it. They propose a new method called TRACE that looks for additional evidence in the whole video to confirm if the snippet really matches the query. This approach improves accuracy on several video search tests by carefully verifying the connection between the query and video frames before choosing the final moment.
Video retrievalPartially relevant videoQuery conditioningEvidence verificationTemporal selectionLocal and global featuresScore calibrationUntrimmed videosBenchmark datasetsActivityNet Captions
Authors
Shuaiqi Cheng, Siyu You, Yanbi Wu, Yuxi Chen, Jiahao Zhang, Xuming Hu
Abstract
Partially Relevant Video Retrieval (PRVR) retrieves untrimmed videos when queries describe only short moments. Although recent methods improve local representations, uncertainty modeling, and global context, final ranking often still trusts the strongest local response; a coincidentally similar fragment can therefore produce an unsupported peak. We identify this failure as the query-agnostic concentration bottleneck and propose TRACE, a score-level evidence verification operator for PRVR. Given a query and global video registers, TRACE activates query-relevant registers, routes their support to frame-level evidence, and smoothly marginalizes alternative query-to-register-to-frame paths before localized temporal selection. Unlike representation-level feature fusion, TRACE uses this evidence only as a query-conditioned residual calibration of the original local score. On ActivityNet Captions, Charades-STA, and TVR, TRACE achieves the best SumR on all three benchmarks and improves the DreamPRVR backbone by 1.2, 1.1, and 1.5 points, respectively. Ablation, routing-corruption, hard-negative, and cross-backbone transfer analyses support the interpretation that the gains arise from query-conditioned evidence verification rather than a generic score offset.