TubeLite: Lightweight Multi-Actor Spatio-Temporal Action Detection
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem of detecting actions in videos by tracking people over time and identifying when actions start and end. They note that existing methods can be complicated and slow because they use heavy transformers or optical flow calculations. Instead, the authors propose TubeLite, a simpler and faster method that keeps track of people consistently across frames and focuses on accurate action timing without needing complex computations. Their approach performs better on some standard video action datasets while using fewer resources.
spatio-temporal action detectionaction tubestemporal localizationvideo-mAPtransformersoptical flowactor trackingtemporal consistencybounding boxesfeature extraction
Authors
Ali Soltaninezhad, Melissa Cote, Alejandro Rico Espinosa, Tunai Porto Marques, Alexandra Branzan Albu
Abstract
Spatio-temporal action detection in videos requires jointly localizing actors in space and identifying action boundaries over time. A common challenge is constructing temporally stable action tubes, as frame-level detectors often suffer from jitter, fragmentation, and imprecise temporal localization. Many recent approaches address this by introducing heavy spatio-temporal transformers or optical-flow-based pipelines, leading to high computational cost and limited scalability. We propose TubeLite, a lightweight framework for spatio-temporal action detection that focuses on stable tube construction and boundary-aware temporal modeling. TubeLite represents each actor as a tube, defined as a sequence of bounding boxes associated with a single actor over time, and explicitly enforces temporal consistency at both the spatial and semantic levels. The method combines low-jitter actor detection, Gaussian-weighted actor feature extraction, efficient short-term temporal propagation, and a boundary-focused temporal prediction head, while avoiding optical flow and large-scale temporal attention. Despite its compact design, TubeLite achieves strong video-level localization performance. It improves Video-mAP@0.5 by 4.5 and 7.1 percentage points over the best compared method on the MultiSports and UCF101-24 datasets, respectively, with substantially fewer parameters and floating-point operations than transformer-based alternatives, demonstrating that effective spatio-temporal action detection can be obtained through principled, lightweight temporal modeling.