Lidar segmentation improves across new sensors without extra data
Robust LIDAR Sensor-to-Sensor Domain Generalization via Accumulated-Point Scene Completion
Computer Vision and Pattern Recognition
Summary
LIDAR sensors help machines understand 3D scenes by sending out laser pulses, but different sensors 'see' the world differently, making it hard for AI models to work well on new sensors without retraining. The authors propose a training method that teaches models to predict both what is in a single scan and what the full scene looks like, using combined information from many scans during training but only the single scan at test time. This approach helps the model generalize better to new, unseen LIDAR sensors without needing new data or labels. Their method also does not slow down the model when it's running in the real world.
What this means in practice
- •For autonomous vehicle engineers: Improve 3D scene understanding on new LIDAR sensor types without collecting or labeling new training data
- •For robotics system developers: Deploy semantic segmentation models that maintain accuracy when switching robot LIDAR sensors without runtime slowdown
Authors
Christoph B Rist, Larissa T Triess, Markus Enzweiler, Dariu M Gavrila
Abstract
We study domain generalization for LIDAR semantic segmentation. Deep neural networks achieve strong performance on 3D point clouds, yet real-world LIDAR exhibits sensor-specific sampling patterns that hinder transfer across datasets. In many applications, performance on completely unseen sensors is crucial. A model should preferably work on a new setup without prior access to its data or labels. To this end, we introduce a source-only training scheme that adds an occupancy prediction auxiliary objective. We aggregate source-domain scans into a simple world model and use it as the supervision signal, replacing single-scan standard segmentation training. A key property of our approach is that it does not alter the input scan. During training and testing the network consumes the same single-frame LIDAR input, only the supervision signal is enriched via the occupancy objective. At deployment, there is no runtime overhead as the auxiliary head can be dropped for pure segmentation, or kept to obtain scene completion alongside semantic segmentation. In an experimental evaluation across distinct LIDAR datasets, we demonstrate that a model's generalization to new domains can be significantly affected by how it is supervised. In certain setups, performance on unseen domains improves without any explicit domain adaptation (no target scans, no target labels, no sensor priors) and without prediction refinement. At the same time, our method maintains competitive source-domain segmentation performance and incurs no additional inference cost.