Multi-view event camera projection improves tracking of multiple objects
MvT: Event-based Multi-view Projection for Multiple Object Tracking
Computer Vision and Pattern Recognition
Summary
Tracking multiple moving objects using special event cameras is tricky because these cameras send data differently than normal cameras. The authors found that looking at this event data from several slightly different views helps tell objects apart, even when they move close together. Their method turns the event data into multiple kinds of 2D views and then combines these views to track objects more accurately without needing training data. This approach works better than previous ones, especially when objects are small or overlap, and it can be used where labeled examples are not available.
What this means in practice
- •For robotics developers: Track several moving objects in real time from event camera data, improving navigation and interaction in crowded environments.
- •For surveillance system engineers: Improve multi-object detection and tracking accuracy using event cameras when objects overlap or move close together, without requiring training data.
Authors
Muxi Zha, Banglei Guan, Minzu Liang, Zibin Liu, Yang Shang, Qifeng Yu, Laurent Kneip
Abstract
Event cameras are increasingly used for Multiple Object Tracking (MOT), but their asynchronous event output often requires specialized methods. Existing processing methods primarily follow two paradigms, pseudo-frames and event-by-event. The former is the prevailing approach since its data format aligns with images, making image-based techniques applicable. However, it suffers from tracking failures when trajectories overlap or are spatially close on pseudo-frames. Facing this challenge, we propose a multi-view pipeline, Multi-view Tracking (MvT), which preserves the 2D data format to leverage image-based techniques directly while introducing additional spatio-temporal views to resolve tracking ambiguities in a single view. MvT comprises a Multi-view Projection (MvP) module and a Multi-view Fusion (MvF) stage. MvP encodes events into three complementary spatio-temporal views while mitigating the pattern discretization. Within MvF, multi-view results are unified into a 3D coordinate system, and tracklets are associated through an optimization model subject to specific criteria combination. Evaluations on four datasets, including our self-collected Small Objects Dataset (SOD), show that MvT seamlessly integrates image-based methods and outperforms existing non-learning and learning trackers in generalized scenarios, and effectively resolves the single-view tracking ambiguities. Being training-free, MvT is applicable when ground-truth annotation is infeasible, thereby highlighting its practical, data-efficient potential. Code is available at https://github.com/zhazhabiu/MvTracking.