Synthetic training improves detection of real urban LiDAR objects
Solution for UCF UrbanTwin LUMPI Track: Sim-to-Real Urban LiDAR 3D Object Detection
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Detecting objects like cars and pedestrians from LiDAR scans is harder when the system is trained on fake data but tested on real data. The authors combine several techniques to make their detector better at this challenge by adjusting training data, using different detectors for various object types, and smartly merging their outputs. They also improve how realistic the synthetic data looks. Their approach works better than simpler methods in detecting real objects from synthetic training.
What this means in practice
- •For autonomous vehicle engineers: Improve vehicle perception systems that must be trained on synthetic urban LiDAR data and yet perform well on real-world scenes.$Commercial implications: Enables commercial self-driving systems to reduce reliance on costly real-world annotated LiDAR data while maintaining detection quality.
- •For robotics software developers: Build robust 3D object detectors for urban robots that face variations between simulated training and real sensor inputs.
Authors
Pu Luo, Cong Xu, Yumei Li, Kexin Zhang, Licheng Jiao, Wenping Ma, Lingling Li
Abstract
We present our solution to the LUMPI track of the UCF UrbanTwin Sim2Real LiDAR Challenge at the 6th DriveX Workshop, ECCV 2026. The detector must be trained only on synthetic data and is evaluated on 50 held-out real LiDAR frames; a separate 50-frame synthetic submission is evaluated for point-cloud realism. Our method addresses the Sim2Real gap at three levels. First, we align synthetic scans to the 50k-point test density and build a 30k-record training pool using UT-LUMPI geometry, RangeLDM-based sampling diversification, rare-class copy-paste, and pedestrian-oriented augmentation. Second, complementary DSVT detectors and Car/Bus PointPillars specialists are trained under the same synthetic-only constraint. Third, predictions are integrated by class-aware routing, asymmetric agreement fusion, constrained residual-recall supplementation, class-coverage auditing, and selective box-size calibration. The realism branch is optimized independently with radial-density matching, weak affine calibration, and calibrated set mixing. The final submission obtains a Combined Score of 0.4692, a Detection Score of 0.1797, a Realism Score of 0.9035, and 3D mAP@0.5 of 0.1258.