Effective synthetic data methods improve urban LiDAR 3D object detection
Solution for UCF UrbanTwin LUMPI Track: Sim-to-Real Urban LiDAR 3D Object Detection
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Detecting objects like cars and pedestrians in 3D from LiDAR scans usually requires real-world data, but this study focuses on training models using only simulated (synthetic) data. To bridge the gap between simulated and real data, the authors use several techniques including making synthetic data resemble real test scans, specialized detectors for vehicle types, and smart ways to combine predictions. They also develop methods to make the simulated point clouds look more realistic. Their approach improves detection accuracy and the realism of synthetic LiDAR data on real urban scenes.
LiDARsynthetic data3D object detectionSim2Realpoint clouddata augmentationdeep learningurban environmentspointpillars3D mAP
Authors
Pu Luo, Cong Xu, Yumei Li, Kexin Zhang, Licheng Jiao, Wenping Ma, Lingling Li
Abstract
We present our solution to the LUMPI track of the UCF UrbanTwin Sim2Real LiDAR Challenge at the 6th DriveX Workshop, ECCV 2026. The detector must be trained only on synthetic data and is evaluated on 50 held-out real LiDAR frames; a separate 50-frame synthetic submission is evaluated for point-cloud realism. Our method addresses the Sim2Real gap at three levels. First, we align synthetic scans to the 50k-point test density and build a 30k-record training pool using UT-LUMPI geometry, RangeLDM-based sampling diversification, rare-class copy-paste, and pedestrian-oriented augmentation. Second, complementary DSVT detectors and Car/Bus PointPillars specialists are trained under the same synthetic-only constraint. Third, predictions are integrated by class-aware routing, asymmetric agreement fusion, constrained residual-recall supplementation, class-coverage auditing, and selective box-size calibration. The realism branch is optimized independently with radial-density matching, weak affine calibration, and calibrated set mixing. The final submission obtains a Combined Score of 0.4692, a Detection Score of 0.1797, a Realism Score of 0.9035, and 3D mAP@0.5 of 0.1258.