Video highlight detection improves with temporal structure aware compression
SGWIB:Sliced Gromov-Wasserstein Information Bottleneck for Video Highlight Detection
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Video highlight detection tries to find the most interesting parts of a video. The researchers introduced a new method that compresses video information while keeping the order and relationship between scenes. This helps the detection system know which parts are important, especially in sports videos. They tested it on two sports video datasets and saw better results than previous methods. Their approach also reduces errors caused by background details unrelated to highlights.
What this means in practice
- •For video platform developers: Improve automatic highlight clips from sports videos by preserving temporal scene relationships during compression.
- •For sports analytics teams: Generate accurate segment-level highlight scores that separate key events from contextual noise in game footage.
Authors
Hanjuan Huang, Yung-Chieh Yeh, Hsing-Kuo Pao
Abstract
Video highlight detection aims to identify temporally important segments that capture the most informative or engaging events in a video. Reliable prediction therefore requires not only discriminative segment representations but also preservation of the temporal relationships among neighboring and distant segments. The information bottleneck principle has proven effective for learning compact and task-relevant representations, yet it has not been explored for video highlight detection, and applying conventional formulations directly would overlook inter-segment relational structure and distort highlight relevant temporal organization during compression. We therefore introduce the Sliced Gromov-Monge Gap (SGMG), a structure aware regularizer that measures the excess relational distortion induced by a prescribed source-to-bottleneck mapping relative to an optimal sliced structural correspondence. Building on SGMG, we develop SGWIB, an information-bottleneck framework for single-modal video highlight detection that learns compact bottleneck representations while preserving inter-segment temporal structure. We further introduce Home-Away-Related Contextual Pseudo-Labels and a contextual disentanglement module that reduce sports-specific contextual bias by separating highlight oriented information from contextual patterns. Experiments on MrHiSum and MoSu show that SGWIB attains the best Kendall's tau, Spearman's rho, mAP@50, and mAP@30 among the compared single-modal methods on both datasets. On MrHiSum, the visual model improves the strongest previous results by 0.031, 0.031, 0.87, and 0.75 on these four metrics, respectively. These results show that structure-aware information-bottleneck regularization combined with contextual disentanglement improves segment-level highlight prediction.