Cooperative system improves 3d tracking of multiple objects across views
FFBL-Coop: Association-Decoupled Cooperative 3D Multi-Object Tracking
Computer Vision and Pattern Recognition
Summary
Tracking multiple objects in 3D space from different viewpoints can be tricky because the system must figure out which observations belong to which objects over time. The authors found that mixing decisions about matching objects and combining their features all at once causes errors due to view differences and misalignment. They introduced a new approach called FFBL-Coop that first decides which objects to admit from other perspectives and then manages identities separately. This method uses a step-by-step process to allocate queries, combine features, and maintain consistent tracking, improving accuracy and reducing communication data without losing performance.
What this means in practice
- •For autonomous vehicle teams: Improve 3D tracking accuracy across vehicles by managing object identities and fused features separately to handle view differences and alignment issues.
- •For robotics fleet operators: Enhance coordinated tracking of multiple objects in shared environments by using efficient methods to combine sensors’ data and maintain consistent object identities.
Authors
Haoxin Wu, Xiaokai Bai
Abstract
Cooperative 3D tracking must integrate complementary observations across agents and time while maintaining consistent identities. When evidence integration and identity inheritance share a matching decision, errors arising from cross-view appearance differences and spatial misalignment can compromise both feature fusion and track continuity. We propose FFBL-Coop, a fuse first, bind later framework that separates instance admission from identity management. Confidence-ranked Slot Admission (CSA) allocates cooperative queries to available ego slots using confidence and spatial proximity. Unified Representation Aggregation (URA) uses cooperative semantic features and aligned anchors to guide ego-feature retrieval, refining the augmented query bank within a shared transformer decoder. After refinement, Cooperative-Priority Identity Anchoring (CPIA) combines learned association with persistent mappings to establish accepted identity assignments across frames. A shared codebook reduces transmitted payload while retaining AP and AMOTA close to the uncompressed variant. FFBL-Coop achieves AMOTA/AP of 0.611/0.548 on V2X-Seq and 0.688/0.653 on Griffin-25M. Code will be released.