Multi camera 3D tracking improved by explicit ID prediction
Online Multi-Camera 3D Tracking via ID Prediction over Recurrent Sparse Queries
Computer Vision and Pattern Recognition
Summary
Tracking objects across multiple camera views in 3D can be tricky because identities get lost when switching views. The authors propose a method that explicitly predicts object IDs using a memory of past trajectories, improving how accurately objects are tracked over time. They tested this on a challenging public benchmark and showed better identity tracking accuracy than previous approaches. This helps keep better track of objects moving through many camera views at once.
What this means in practice
- •For video surveillance operators: Improve real-time multi-camera tracking accuracy across multiple viewpoints by explicitly maintaining global identities in crowded scenes.
- •For autonomous vehicle developers: Enhance 3D tracking of pedestrians and vehicles from multiple sensors combined into a unified world frame for safer navigation decisions.
Authors
Pragyan Shrestha, Haruto Nakayama, Atom Scott
Abstract
Online multi camera 3D tracking must maintain scene global identities across synchronized views, yet query-based trackers carry these identities only implicitly in the instance bank, where they fragment upon query interruption. We present an online architecture that recovers association accuracy by predicting IDs explicitly over recurrent sparse queries. An outside-in Sparse4D detector fuses calibrated views into world frame 3D detections while propagating a sparse query bank, and a causal MOTIP ID decoder associates detections against a finite trajectory memory. We adapt MOTIP's relative-ID prediction and recycled slot runtime to globally fused 3D observations, and introduce metric spatial gating and proximity based newborn recovery. On the official 2026 AI City Challenge Track 1 test set, our method raises HOTA from 29.63 with native instance bank identities to 38.01, primarily through an AssA increase from 20.83 to 31.10, and ranks third on the public leaderboard. Full-sequence validation over all 9,000 frames of each scene shows that decoupled ID training improves HOTA over native identities, whereas continuing detector training alongside the detached ID objective produces scene-dependent gains and losses.