Self supervised video synthesis boosts repetitive action counting accuracy
TReViS: Temporal Repetition Structure Aware Video Synthesis for Self-supervised Repetitive Action Counting
Computer Vision and Pattern Recognition
Summary
Counting repeated actions in videos usually needs lots of time-consuming labels. The authors propose TReViS, a method that learns from unlabeled videos by finding patterns of repetitions and creating new training videos with similar cycles. This lets existing counting models train without labeled examples. Their approach performs better than other label-free methods and nearly matches models trained with labels.
What this means in practice
- •For video analytics developers: Train action counting models without labeled videos by synthesizing realistic repetitive action sequences.
- •For sports performance analysts: Analyze athlete training videos to count exercise repetitions without needing manual annotation.
Authors
Fanqi Yu, Shengming Ma, Stefano Fiorini, Vito Paolo Pastore, Xuan Qi, Vittorio Murino, Cigdem Beyan
Abstract
Fully supervised repetitive action counting (RAC) has achieved strong performance, but requires dense temporal annotations that are costly and difficult to scale. We propose TReViS, a self-supervised video synthesis framework that enables training RAC models without any repetition labels. TReViS estimates the underlying temporal repetition structure of an unlabeled video via a Temporal Self-Similarity Matrix, infers its cycle statistics, and synthesizes new training sequences that preserve realistic repetition patterns while introducing controlled temporal variability. These synthesized videos are paired with pseudo-labels and used to train existing RAC architectures from scratch. Across multiple datasets and backbones, TReViS consistently outperforms prior self-supervised methods and achieves performance competitive with several supervised baselines, while remaining fully label-free, demonstrating the effectiveness of structure-aware video synthesis for label-free RAC. The source code is available at https://github.com/yfqi/TReViS.