Detector choice and scale shape multi-pedestrian video tracking quality
The Impact of Detector Architecture, Model Scale, and Scene Density on Multi-Pedestrian Tracking
Computer Vision and Pattern RecognitionMachine Learning
Summary
Tracking many people in videos is hard, especially when lots of people are close together or moving fast. The authors studied how different video detectors, their sizes, and crowded scenes affect how well pedestrians are tracked over time. They tested several detector and tracker combinations on popular datasets and found that some detectors work better in dense scenes but need more computing power, while others offer a good balance of speed and accuracy. Their work helps people pick the best tools depending on what is more important for their project.
What this means in practice
- •For surveillance system engineers: Optimize pedestrian tracking performance by choosing detectors that balance accuracy and speed based on crowd density in video monitoring.
- •For traffic management operators: Improve tracking quality of people flows in crowded transport hubs by selecting detector and tracker combos suited for high-density scenes.
Authors
Pedro Manoel Herminio Alves, Dalton Cézane Gomes Valadares, Cláudio de Souza Baptista
Abstract
Pedestrian tracking in video is important for applications such as surveillance and scene analysis, but it becomes more challenging in crowded scenarios with frequent occlusions and rapid motion. This paper analyzes how detector architecture, model scale, scene density, and inference regime affect the tracking of multiple pedestrians. We compare the YOLO11, RT-DETR, and YOLO26 detector families with the ByteTrack and BoT-SORT trackers on sequences from the MOT17, MOT20, and SOMPT22 benchmarks. For YOLO26, we also compare traditional overlap filtering via Non-Maximum Suppression (NMS) with end-to-end inference. The RT-DETR family achieved the highest mean scores on the HOTA tracking quality metric across all three benchmarks with both trackers, with a more pronounced advantage on MOT20, albeit at a higher computational cost. On MOT17 and SOMPT22, YOLO26 variants, especially YOLO26-L with NMS, offered favorable combinations of quality and speed compared with RT-DETR. In the YOLO26 inference comparison, the end-to-end mode did not yield consistent tracking gains over the NMS-based regime. Taken together, these results provide a practical guide for selecting detectors and trackers for this task based on application priorities.