Multi-object tracking improved by automating hyperparameter tuning
Revisiting Multi-Object Tracking Baselines: Hyperparameter Optimization with Multi-Fidelity Greedy Coordinate Search
Computer Vision and Pattern Recognition
Summary
Tracking multiple objects in videos usually depends on settings chosen by hand, which can be slow and inconsistent. The authors showed that automatically tuning these settings with careful search methods can lead to better tracking results. They introduced a new search technique called Multi-Fidelity Greedy Coordinate Search, which tests settings on small parts of the data before expanding to the full set. This approach often found better settings faster than previous methods and improved tracking accuracy. They also shared their code and tuned settings to help others start from these better baselines.
What this means in practice
- •For computer vision engineers: Use the authors' automated hyperparameter tuning methods to improve multi-object tracking in video analysis systems, achieving better accuracy without manual trial and error.
- •For autonomous vehicle developers: Apply systematic hyperparameter optimization to tracking components for more reliable pedestrian and vehicle tracking in self-driving cars.
Authors
Momir Adžemović
Abstract
Multi-object tracking (MOT) is dominated by the tracking-by-detection paradigm, whose methods typically rely on a small set of hyperparameters that are conventionally chosen by hand. Tuning them requires repeated expert-guided experimentation, while the procedures used to select reported values are often not systematically evaluated or fully documented. Hyperparameter optimization (HPO) automates this process, yet it remains rarely used in MOT, and existing studies applying HPO to MOT predate modern deep-detector-based trackers and HOTA evaluation. We systematically apply HPO across two datasets and four tracking-by-detection methods. We also propose Multi-Fidelity Greedy Coordinate Search (MFGCS), which optimizes one hyperparameter at a time by first evaluating candidate values on a small subset of scenes and re-evaluating only promising candidates on the full dataset. Across all eight tracker-dataset combinations, the Tree-structured Parzen Estimator (TPE) and MFGCS outperform both our hand-tuned configurations and the corresponding published results, with improvements of up to 4.38 and 16.05 HOTA points, respectively. MFGCS also reaches a predefined HOTA target faster than TPE in seven of the eight combinations. Within each tracker-dataset pair, all optimizers share the same search space and evaluation pipeline, isolating the effect of the search strategy. We release the code and tuned configurations to enable future work to compare against systematically optimized rather than default or manually tuned baselines.