Hierarchical sparse graph enables accurate event based tracking across time

Hierarchical Sparse Graph for Multi-dimensional Assignment in Event-based Tracking

Computer Vision and Pattern Recognition

Summary

Tracking objects that generate event data is hard, especially when objects disappear or get missed for a while. The authors handle this by linking object tracks over multiple time batches at once, instead of just nearby times. Their method treats the problem like connecting a fuzzy web of possibilities, allowing multiple matches without strict rules. This approach uses a special graph and smart grouping techniques to find the best matches without needing training. Tests show it works better than existing event and video tracking methods, especially in tough cases.

What this means in practice

  • For autonomous vehicle engineers: Improve long term object tracking accuracy using event sensors when objects temporarily disappear or get occluded.
  • For drone navigation teams: Enhance drone tracking systems for complex environments by associating detection results over extended time intervals.

Authors

Muxi Zha, Yang Shang, Wenlong Zhang, Qifeng Yu, Banglei Guan

Abstract

Existing event-based tracking methods typically operate in an online manner, associating tracklets across adjacent batches. However, this limited temporal window often leads to failures in challenging scenarios, such as long-term occlusions or missed detections, where broader temporal context should be incorporated for high-precision tracking. To achieve robust and accurate tracking, this paper concentrates on the event-based multi-dimensional assignment that associates tracklets across multiple batches simultaneously. Unlike image-based detectors, event-based detectors can generate multiple correct yet spatially distinct tracklets for a single object per batch. This violates the one-to-one rule in classic data association, making event-based association an underdetermined many-to-many matching problem. To the best of our knowledge, this is the first work to tackle this NP-hard problem. To address its underdetermined nature, we formulate the assignment problem via fuzzy logic, where pairwise tracklet correlations are quantified through multiple long-range kinematic metrics. This fuzzy design eliminates the need for hard tracking thresholds and enables multiple heterogeneous metrics to be integrated into a unified correlation. By treating tracklets as nodes and their fuzzy correlations as edges, we construct a Hierarchical Sparse Graph (HSG). Building upon the HSG, we pose assignment as a graph partition task, and employ a heuristic solution that measures vertex affinity for partitioning adaptively. The entire pipeline is training-free. Comprehensive experiments on two challenging event datasets, eTram and Ev-UAV, show that our method outperforms existing event-based and most image-based association models. Particularly, it achieves over 2$\times$ the performance of OC-SORT across all metrics in the many-to-many association scenario.