CurvSpec improves finding short video moments matching text queries
CurvSpec: Adaptive Multi-Curvature Learning for Partial Relevant Video Retrieval
Computer Vision and Pattern Recognition
Summary
Sometimes, finding a short moment in a long video that matches a text description is very hard because the important part is very brief and buried in lots of unrelated content. The researchers propose CurvSpec, which adapts how videos are represented by learning different geometric spaces, letting the system better handle videos that vary from simple events to complex scenes. CurvSpec also breaks videos into key segments based on content and matches queries to the closest segment, reducing confusion from irrelevant parts. Tests show this approach outperforms previous methods in finding relevant moments across several video datasets.
What this means in practice
- •For video platform engineers: Enable better search for brief relevant moments inside long videos without needing manual time labels by adapting how video data is represented geometrically.
- •For content moderation teams: Improve detection of specific scenes from text descriptions in untrimmed videos for faster review and filtering of inappropriate content.
Authors
Zhen Liu, Letian Li, Jinpeng Wang, Shuzhao Xie, Yuzhi Huang, Jingyan Jiang, Zhi Wang
Abstract
Partially Relevant Video Retrieval (PRVR) seeks to retrieve untrim-med videos containing a moment that matches a text query, without temporal annotations. The relevant moment may last only seconds within a video spanning several minutes, creating an extremely low signal-to-noise ratio that makes PRVR more challenging than standard full-video retrieval. This task presents two intertwined challenges: (1) signal dilution, where coarse global representations blur the brief relevant signal into the dominant irrelevant surroundings;(2) curvature rigidity, where embedding all videos in the same fixed-geometry space distorts representations for videos that range from flat atomic events to deep compositional hierarchies. Existing PRVR methods have improved moment selection and cross-modal matching, but they still typically encode all videos in a single fixed-curvature retrieval space, limiting their ability to model diverse video structures. To address both challenges, we propose CurvSpec, a framework that learns content-adaptive curvature for video retrieval representations rather than imposing a fixed geometric prior. CurvSpec processes features through parallel Euclidean and hyperbolic attention layers, with independently learned curvatures assigned to the hyperbolic layers, and a content-aware fusion mechanism routes each input to its most suitable geometric regime. To further suppress signal dilution, CurvSpec represents each video with semantic centroids whose number is determined by the video's content complexity, projects them onto the learned manifold, and matches each query against its nearest centroid by geodesic distance. Experiments on ActivityNet Captions, TVR, and Charades-STA demonstrate state-of-the-art retrieval performance.