Visual object tracking improves with continuous spatial and time context
Continuous Token-Level Spatio-Temporal Context Modeling for Visual Object Tracking
Computer Vision and Pattern Recognition
Summary
Many computer programs track objects in videos by looking at snapshots taken at specific times, but this can miss important continuous information. The authors developed a new method called TLCTrack that keeps updating and focusing on important parts of the scene over time to track objects more accurately. Their approach includes techniques to ignore distracting background areas and better capture details across both space and time. Tests show this method works better than many existing ways to follow moving objects in video. They also made their code available for others to use.
visual trackingspatio-temporal contexttoken-level modelingmasked unidirectional attentionsalient tokensspatial representationbackground suppressionobject representationtracking benchmarks
Authors
Ding Xia, Meiqin Liu, Jing Zhou, Jian Lan
Abstract
Spatio-temporal context has become increasingly crucial for visual tracking. However, most existing approaches extract spatio-temporal cues via discrete sampling strategies, which inherently deviate from the continuity of spatio-temporal context, thereby deteriorating tracking performance. To address this challenge, we propose TLCTrack, a novel tracking framework that models token-level spatio-temporal context through continuously updated salient tokens, enabling more accurate target representation. Specifically, TLCTrack incorporates three components: Masked Unidirectional Attention (MUA), Spatial Salient Token Collection (SSTC), and Temporal Salient Token Bank (TSTB) modules. By explicitly integrating spatio-temporal context, MUA extracts discriminative targetaware spatial features in the search region. To avoid the negative impact of background on feature learning, SSTC progressively suppresses background interference, thereby enhancing target spatial representation. Finally, TSTB captures high-quality spatio-temporal information through continuous salient token updates. Extensive experiments on five benchmarks demonstrate that our method achieves superior performance over state-of-the-art trackers. Code and models are available at https://github.com/xiading123/TLCTrack.