VLA-ReID: Video-Level Association for Re-Identification in Multi-Object Tracking with Highly Similar Objects
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
The authors focus on tracking many similar-looking objects, like bees, in videos while keeping their identities correct over time. They point out that existing methods train models to recognize objects one at a time, but during tracking must match many objects at once, which causes mistakes. Their solution, called VLA-ReID, treats the problem as matching sets of objects from the past and present all at once, making it easier to keep identities correct. They also introduce techniques to highlight unique features of each object without needing extra labeled data. Tests on bee videos show their method reduces identity mix-ups and improves tracking accuracy.
Authors
Yanrong Qin, Xiaoyan Cao, Yao Yao
Abstract
Multi-object tracking (MOT) aims to localize multiple objects in videos while preserving their identities over time. Long-term identity preservation remains difficult when objects are small, densely distributed, and highly similar in appearance, as in bee swarm scenes. Existing trackers rely on re-identification (re-ID) models trained through single-instance assignment (instance-level querying). At inference, however, MOT requires global assignment between multiple trajectories and detections, corresponding to video-level querying. This training-inference mismatch can cause identity switches among visually similar objects. Existing approaches also often require substantial additional annotations to enhance appearance discrimination. We propose Video-Level Association re-ID (VLA-ReID), which reformulates re-ID as video-level association modeling. It uses aggregated historical trajectory features as queries and all current-frame detections as candidates, enabling direct optimization of their global association at each frame. In addition, Frame-Common Appearance Estimation (FCAE) estimates a common appearance direction from current-frame detections, while Common-Appearance Suppression (CAS) removes the corresponding component along this direction from trajectory and detection features. This amplifies discriminative differences among highly similar objects without additional annotations. Experiments on BEE24 show that VLA-ReID improves HOTA by 1.1, MOTA by 0.3, AssR by 2.6, AssA by 0.7, and IDF1 by 0.8 over state-of-the-art trackers, while reducing identity switches by 28%. These results demonstrate the effectiveness of video-level re-ID modeling for appearance-based association in MOT.