Rgb and thermal images better aligned and combined using frequency equalization
SAGE: Source-Anchored Guidance via Frequency Equalization for Hierarchical RGB-T Alignment and Fusion
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Combining normal color pictures (RGB) with thermal images can be tricky because the pictures might not line up well and show mixed-up details. The authors created a new method called SAGE that helps match and blend these two kinds of images more smoothly by focusing on different layers of image details using frequencies. This approach keeps important information from both images while correcting misalignments and combining their details effectively. Tests showed that SAGE works well even when the pictures don’t line up perfectly.
What this means in practice
- •For security system designers: Improve the quality of fused RGB and thermal images to enhance object detection in low visibility conditions.
- •For autonomous vehicle engineers: Provide more reliable alignment and fusion of RGB and thermal camera inputs to improve perception accuracy in diverse environments.
Authors
Timing Li, Yiming Sun, Boan Tao, Xiyuan Gao, Haifang Cao, Pengfei Zhu
Abstract
Spatial misregistration and cross-modal discrepancies often cause ghosting, structural blurring, and content imbalance in RGB-T fusion. Existing methods typically decouple appearance adaptation, geometric alignment, and information fusion, limiting dependency propagation across stages. We propose Source-Anchored Guidance via Frequency Equalization for Hierarchical RGB-T Alignment and Fusion (SAGE), a unified framework integrating frequency equalization, hierarchical alignment, and subband fusion. SAGE employs invertible joint encoding and source-specific low-frequency modulation to derive structural and gain guidance while preserving source information. Hierarchical frequency collaborative alignment estimates global affine geometry from low-frequency approximations and transfers geometric and contextual cues to high-frequency correlation reasoning for reliability-aware residual refinement. Guided subband fusion jointly aggregates the aligned frequency coefficients under propagated source and alignment guidance, coordinates complementary low- and high-frequency information, and reconstructs the fused image through the inverse wavelet transform. Extensive experiments on RGB-T datasets with real-world and synthetic misalignments demonstrate consistently competitive performance in alignment and fusion, validating the effectiveness of source-anchored guidance for weakly registered RGB-T images.