Point supervision method improves video action detection accuracy
Learning High-Quality Pseudo-Labels for Point-Supervised Temporal Action Localization
Computer Vision and Pattern Recognition
Summary
Identifying when and where actions happen in long videos is hard to label precisely. This paper presents a new method that uses limited point-level annotations to generate better guesses for action boundaries during training. The method combines initial confidence with additional cues to pick good action segments, then refines these guesses by comparing multiple candidates and updating as the model learns. This approach leads to more accurate detection of actions in videos, improving performance on standard benchmarks.
What this means in practice
- •For video analysis teams: Use improved pseudo-labeling to more accurately identify action segments in videos with minimal manual annotation.
- •For surveillance system developers: Enhance automated detection of specific activities in long surveillance footage where detailed labeling is costly.
Authors
Zhenying Fang
Abstract
Point-supervised temporal action localization (PTAL) aims to recognize and localize action instances in untrimmed videos using only point-level annotations. Existing pseudo-label-based methods have made substantial progress in PTAL. However, confidence-based initialization can overlook well-localized proposals. Meanwhile, existing methods either keep pseudo-labels fixed during detector training or update them without fully exploiting alternative boundary evidence from the original proposal pool. These limitations constrain supervision quality, thereby hindering detection performance. To learn high-quality pseudo-labels, we propose a framework that integrates prior-guided initialization with consensus-guided refinement during detector training. Our framework comprises two modules: Prior-Guided Initialization (PGI) and Consensus-Guided Refinement (CGR). PGI combines proposal confidence with point-derived priors to establish instance-specific reliability for initial pseudo-label selection and candidate retention. CGR continually uses the retained original candidates to construct boundary targets, combining their consensus with evolving detector predictions to refine pseudo-labels throughout training. Experiments on four benchmarks demonstrate that our method achieves competitive detection performance compared with existing methods. In particular, our method achieves 68.5\% average mAP over tIoU thresholds 0.1--0.7 on THUMOS14, surpassing PseR by 3.7 percentage points. Code is available at https://github.com/zhenyingfang/LHQP