HiPFuse improves real-time lidar semantic maps on edge devices
HiPFuse: Real-Time LiDAR Semantic Segmentation for Robotic Perception on Edge Devices
Computer Vision and Pattern RecognitionRobotics
Summary
Real-time robotic perception faces challenges in accurately understanding 3D scenes from lidar data with limited computing power. The authors designed HiPFuse, which smartly combines detailed points and bird's-eye view features while keeping vertical layers separate. This method runs efficiently on small edge devices like the Jetson AGX Orin and improves semantic maps for driving and indoor environments. The authors show that using HiPFuse reduces errors in robot positioning and better recognizes continuous road surfaces.
What this means in practice
- •For robotics system builders: Integrate HiPFuse to enhance semantic understanding and localization accuracy in mobile robots operating with constrained onboard computing.
- •For autonomous vehicle developers: Deploy HiPFuse on edge hardware to improve real-time semantic mapping of outdoor environments, supporting tasks like loop closure and trajectory correction.
Authors
Haiyang Wu, George Vosselman, Ville Lehtola
Abstract
Real-time robotic perception requires accurate LiDAR semantics under limited resources. Bird's-eye-view (BEV) aggregation is efficient but can mix differently labeled surfaces in columns. We present HiPFuse, a point-BEV fusion framework that exchanges point and height-partitioned BEV features across horizontal scales, combining point-wise detail with 2D spatial context from one LiDAR scan. Fixed height assignments preserve separate vertical aggregation groups, with auxiliary supervision of initial BEV features using targets from the same 3D cells. A dual-scale BEV context module adds broader context from a coarser grid. On SemanticKITTI and MCD, HiPFuse achieves 63.7% and 62.5% mIoU at 14.2 and 22.4 FPS on Jetson AGX Orin, respectively, exceeding the 10 Hz scan rate. Separate training and evaluation show indoor applicability. Replacing only the baseline semantic front end with HiPFuse in fixed robotic systems reduces odometry absolute trajectory error RMSE by 10.6% on average across two outdoor sequences and increases accepted-loop recall by 8.7 percentage points on SemanticKITTI. Qualitative semantic maps show more continuous road labeling. Our code is available at https://github.com/haiyang2022/HiPFuse.