LiDAR semantic segmentation improved by ray-aligned and scan-aware fusion
RaSFNet: Ray-aligned and Scan-aware Fusion Network for LiDAR Semantic Segmentation
Computer Vision and Pattern RecognitionMachine Learning
Summary
Understanding what objects are in a 3D space from LiDAR scans is important for things like self-driving cars. The authors address problems with current methods that lose geometric details and mix points from different areas when converting 3D data to 2D for analysis. They propose a new network that better keeps track of how LiDAR points are distributed along scanning rays and organizes data according to the scanner's rotation pattern. This improves recognizing different objects by combining point and projected information in a smarter way. Their approach shows competitive accuracy on standard LiDAR datasets used for testing.
What this means in practice
- •For autonomous vehicle engineers: Enhance 3D object recognition in LiDAR data by integrating scanning patterns and point geometry for better scene understanding.
- •For robotics system developers: Improve environment perception in robots relying on LiDAR sensors by using scan-aware fusion methods for clearer semantic maps.
Authors
Chuang Chen
Abstract
LiDAR point cloud semantic segmentation is a fundamental component of 3D perception for autonomous driving and robotics. Projection-based methods map 3D point clouds into 2D representations for feature extraction using 2D networks. However, aggregating multiple points during projection can obscure geometric differences within each frustum, while local feature extraction limits the modeling of spatial relationships across scanning regions. Moreover, proximity in the projected representation does not necessarily imply proximity in 3D space, and interactions between point and projected features may mix features from different ranges, reducing semantic discriminability. To address these challenges, we propose the Ray-aligned and Scan-aware Fusion Network (RaSFNet), which incorporates point cloud geometry and LiDAR scanning characteristics into projection-based semantic segmentation. Specifically, the Ray-Aligned Geometric Distribution Encoding (RAGDE) module characterizes point distributions within each frustum using a ray-aligned coordinate system. To model spatial context, a Scan-Topology-Aware Mamba (STAM) block organizes frustum features according to the LiDAR azimuth sampling pattern and captures long-range dependencies across scanning regions. In addition, a Range-Constrained Point-Frustum Fusion (RCPFF) mechanism uses shared range information to guide feature exchange between point and frustum representations while maintaining spatial correspondence. RaSFNet achieves mIoU scores of 70.2% and 77.7% on the SemanticKITTI and nuScenes validation sets, respectively, indicating competitive segmentation performance compared with the evaluated methods.