SGPDFuse: Semantically-Guided Physics-Disentanglement General Multi-Modal Image Fusion
Computer Vision and Pattern Recognition
Summary
The authors present SGPDFuse, a method for combining images from different types of sensors, like infrared and visible light, into one clear picture that keeps important details and removes noise. They use a technique called Semantic-Physical Parametric Bridge to separate the true scene features from environmental effects by leveraging pretrained vision models. Their Semantic Alignment mechanism ensures that important objects stay clear, and unwanted artifacts are removed. Tests show that their approach works well on different image fusion tasks with a single system.
Authors
Haozhen Wei, Chengjun Jiang, Yutong Guo, Xinrui Ju, Xingyuan Li, Xiang Chen, Jinyuan Liu
Abstract
Multimodal image fusion (MMIF) aims to integrate complementary sensor data into a single representation that preserves intrinsic scene reality while eliminating environmental interferences. Most existing approaches rely on blind feature aggregation, which excels at signal accumulation but fails to distinguish essential content from physical degradations. We propose SGPDFuse, which bridges this gap by mapping inputs into a physics-disentangled structural representation via a Semantic-Physical Parametric Bridge (SPPB) built on pretrained vision foundation models, utilizing the Intrinsic-Variation principle to decouple invariant scene attributes from transient environmental factors. To guide this decomposition, we introduce a Semantic Alignment mechanism: we explicitly anchor the fused representation to salient semantic features in the same foundation model feature space via cosine similarity to preserve critical targets, while enforcing physical texture fidelity through Gram-matrix regularization to strictly eliminate unnatural artifacts. Extensive experiments demonstrate that SGPDFuse achieves state-of-the-art performance across infrared-visible, multi-focus, and multi-exposure benchmarks using a single architecture.