Hyperbolic Distillation: Geometry-Guided Cross-Modal Transfer for Robust 3D Object Detection
2026-05-11 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors developed a new method called HGC-Det to improve 3D object detection by combining features from both images and point clouds more effectively. Their approach includes three parts that refine 3D data using image information, use hyperbolic geometry to better merge different types of features, and fix any loss of spatial details. They tested their method on several indoor and outdoor datasets and found it balances detection accuracy with computational efficiency better than previous methods. This work helps address challenges in mixing different data types for 3D perception tasks.
Cross-modal knowledge distillation3D object detectionPoint cloudImage featuresHyperbolic geometryVoxel optimizationSemantic guidanceFeature aggregationSUN RGB-DKITTI dataset
Authors
Kanglin Ning, Wenrui Li, Houde Quan, Qifan Li, Xingtao Wang, Xiaopeng Fan
Abstract
Cross-modal knowledge distillation has emerged as an effective strategy for integrating point cloud and image features in 3D perception tasks. However, the modality heterogeneity, spatial misalignment, and the representation crisis of multiple modalities often limit the efficient of these cross-modal distillation methods. To address these limitations in existing approaches, we propose a hyperbolic constrained cross-modal distillation method for multimodal 3D object detection (HGC-Det). The proposed HGC-Det framework includes an image branch and a point cloud branch to extract semantic features from two different modalities. The point cloud branch comprises three core components: a 2D semantic-guided voxel optimization component (SGVO), a hyperbolic geometry constrained cross-modal feature transfer component (HFT), and a feature aggregation-based geometry optimization component (FAGO). Specifically, the SGVO component adaptively refines the spatial representation of the 3D branch by leveraging semantic cues from the image branch, thereby mitigating the issue of inadequate representation fusion. The HFT component exploits the intrinsic geometric properties of hyperbolic space to alleviate semantic loss during the fusion of high-dimensional image features and low-dimensional point cloud features. Finally, the FAGO compensates for potential spatial feature degradation introduced by the 2D semantic-guided voxel optimization component. Extensive experiments on indoor datasets (SUN RGB-D, ARKitScenes) and outdoor datasets (KITTI, nuScenes) demonstrate that our method achieves a better trade-off between detection accuracy and computational cost.