Heat conduction approach improves joint classification of hyperspectral and LiDAR data
Toward Interpretable Multimodal Fusion: Heat Conduction Modeling for Hyperspectral and LiDAR Joint Classification
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Combining data from hyperspectral imaging and LiDAR helps better identify land types on Earth. The authors propose a new method inspired by how heat spreads, to mix these two data types more effectively. Their approach captures complex relationships across the data while remaining computationally efficient and easier to understand. Tests on several datasets show their method performs competitively and provides insight into how the data fusion works.
What this means in practice
- •For remote sensing analysts: Improve the accuracy and interpretability of land-cover classification models that integrate hyperspectral and LiDAR data.
- •For geospatial data engineers: Build efficient land mapping tools that handle complex dependencies across spectral and structural data for better environmental monitoring.
Authors
Kan Wei, Jiahui Cui, Jing Yao, Xinyu Zhao, Lei Wang, Pedram Ghamisi
Abstract
The fusion of hyperspectral (HS) and Light Detection and Ranging (LiDAR) data plays a crucial role in enhancing land-cover classification by jointly exploiting spectral, spatial, and structural cues. However, existing multimodal fusion methods still struggle to model long-range dependencies and complex anisotropic interactions while maintaining computational efficiency. This paper introduces M2Heat, a physics-inspired framework that investigates multimodal fusion through the lens of heat conduction. At its core, a physics-driven visual heat conduction module (vHeat) and enhanced Frequency Value Embeddings (FVEs) simulate anisotropic information flow, enabling the capture of global dependencies with sub-quadratic complexity and physical interpretability. This mechanism, combined with a hybrid spatial-frequency fusion strategy named Cross-Frequency Fusion (CFF) module, produces highly discriminative and robust feature representations. M2Heat achieves competitive overall performance on three benchmarks, i.e., Trento, Houston2013, and Augsburg, while providing an interpretable heat-conduction-guided perspective for multimodal feature fusion. These results indicate the potential of heat-conduction-guided neural operators for efficient and interpretable RS multimodal fusion. The source code is publicly available at https: /github.com/Weikan0425/M2Heat_HSI_LiDAR.