LiDAR detection improves urban vehicle and pedestrian tracking
Solution for UCF UrbanTwin V2X-Real Track: Sim-to-Real Urban LiDAR 3D Object Detection
Computer Vision and Pattern Recognition
Summary
Detecting objects like vehicles and pedestrians using 3D LiDAR scanners in cities is tricky because real-world data differs from simulated data. The authors created a system that combines different types of simulated scans to better match real-world conditions. They use specialized parts of the system for different object types and then fuse the results in a smart way to improve accuracy. Their approach works well on a challenging urban dataset, showing that carefully mixing data sources is better than just combining many model outputs randomly.
What this means in practice
- •For autonomous vehicle developers: Improve roadside LiDAR detection systems by training on multiple simulated scan types to better handle real urban scenes and pedestrian shapes.
- •For smart city infrastructure teams: Deploy roadside sensor networks with refined 3D detection models that fuse diverse data sources for more reliable vehicle and pedestrian tracking in cities.
Authors
Pu Luo, Cong Xu, Yumei Li, Kexin Zhang, Licheng Jiao, Wenping Ma, Lingling Li
Abstract
Bridging the simulation-to-reality gap in roadside LiDAR requires addressing several coupled discrepancies, including scene geometry, sampling density, return patterns, and pedestrian scale. This report presents a multi-source collaborative training and class-aware fusion framework for Sim2Real 3D detection. The method organizes digital-twin scans, diffusion-redrawn scans, density-stabilized scans, and pedestrian morphology-aligned samples into a unified training pool with complementary roles. Within a common DSVT detection formulation, source-specialized expert branches preserve those roles while optimizing for the same detection objective. At inference, a predefined class-aware fusion pathway integrates geometry-stable and calibration-aware branches for vehicles, sampling-complementary branches for trucks, and morphology-consistent evidence for pedestrians. A label-free point-cloud center blend then refines geometric localization. On the UrbanTwin V2X-Real hidden test set, the unified system achieves a combined score of 0.7421, with 3D mAP@0.5 of 0.4518 and a realism score of 0.8871. The results indicate that a stable, interpretable collaboration among data sources is more valuable than unconstrained aggregation of model outputs.