Visual tracking improves by modeling continuous spatio-temporal context
Continuous Token-Level Spatio-Temporal Context Modeling for Visual Object Tracking
Computer Vision and Pattern Recognition
Summary
Visual tracking means following a moving object in video frames. Most existing methods look at pieces of the video at set time steps, but this misses the smooth changes that happen naturally over time and space. The authors propose a new way to keep track of important tokens representing the object continuously and update them to better capture its movement and appearance. This approach reduces background noise and improves how the object is recognized and followed in video. They tested their method on many benchmark datasets and found it works better than previous systems.
What this means in practice
- •For computer vision engineers: Implement improved object tracking in videos with continuous spatial and temporal context modeling for more accurate target representation.
- •For surveillance system developers: Enhance real-time tracking accuracy in security cameras by continuously updating key object features across time and space.$Commercial implications: Better tracking algorithms can be sold as part of security solutions for improved monitoring and incident detection.
Authors
Ding Xia, Meiqin Liu, Jing Zhou, Jian Lan
Abstract
Spatio-temporal context has become increasingly crucial for visual tracking. However, most existing approaches extract spatio-temporal cues via discrete sampling strategies, which inherently deviate from the continuity of spatio-temporal context, thereby deteriorating tracking performance. To address this challenge, we propose TLCTrack, a novel tracking framework that models token-level spatio-temporal context through continuously updated salient tokens, enabling more accurate target representation. Specifically, TLCTrack incorporates three components: Masked Unidirectional Attention (MUA), Spatial Salient Token Collection (SSTC), and Temporal Salient Token Bank (TSTB) modules. By explicitly integrating spatio-temporal context, MUA extracts discriminative targetaware spatial features in the search region. To avoid the negative impact of background on feature learning, SSTC progressively suppresses background interference, thereby enhancing target spatial representation. Finally, TSTB captures high-quality spatio-temporal information through continuous salient token updates. Extensive experiments on five benchmarks demonstrate that our method achieves superior performance over state-of-the-art trackers. Code and models are available at https://github.com/xiading123/TLCTrack.