Multi-object tracking improved by clear evaluation of detection methods
Tracking-by-detection in Multi-object Tracking: Survey and Experiments
Computer Vision and Pattern Recognition
Summary
Tracking many moving things in videos at once is important for things like security cameras and self-driving cars. One popular way to do this, called tracking-by-detection, uses separate steps to find objects and then follow them over time. The authors found that past studies often tested improvements in different ways, making it hard to know which ideas really help. They carefully compared different parts of tracking methods using the same tests and datasets to find the best combination. Their work helps create more reliable systems to track multiple objects in real-world situations.
multi-object trackingtracking-by-detectionobject detectiondata associationsimilarity measurementmotion modelscamera motion compensationinterpolation strategiesbenchmark evaluationvideo sequences
Authors
Yujin Yang, Kyujin Shim, Kangwook Ko, Changick Kim
Abstract
Multi-object tracking (MOT) is an essential computer vision task that simultaneously tracks multiple objects in video sequences, with various applications in surveillance, autonomous navigation, and human-computer interaction. The tracking-by-detection (TBD) paradigm, which combines object detection with temporal association, has emerged as a leading approach, driven by innovative algorithms. Despite recent progress, fair evaluation of TBD-based methods remains a challenge. Many studies introduce modules such as similarity metrics, data association strategies, or motion models, but they are often evaluated under inconsistent protocols, with different baseline trackers, hyperparameters, and datasets. Such inconsistencies obscure the genuine contribution of each module and hinder objective comparison. This survey systematically reviews TBD-based MOT techniques, including similarity measurements, data association, camera motion compensation, and interpolation strategies. Starting from a minimal baseline tracker, we fairly evaluate the contributions of each method across diverse datasets and accumulate well-balanced methods. Our findings establish a strong baseline tracker and provide a foundation for the principled design of robust and versatile MOT systems suitable for real-world deployment.