Rgb thermal detection adapts to unreliable sensor data for better results
RA-SOD: Reliability-Aware RGB-T Salient Object Detection under Modality Degradation
Computer Vision and Pattern Recognition
Summary
Detecting important objects using regular and thermal cameras can be tricky when the camera images are blurry, noisy, or unclear. The authors created a system called RA-SOD that learns how reliable each camera is at every moment and uses this to combine their images better. This approach helps the system focus on trustworthy information, improving object detection even when one camera's image is poor. They tested this technique on several datasets and found it performs better than other methods.
What this means in practice
- •For autonomous vehicle engineers: Improve object detection under poor lighting or sensor conditions by adaptively weighting visible and thermal camera inputs.
- •For security system developers: Develop surveillance systems that maintain detection accuracy despite noisy or degraded RGB or thermal video feeds.
Authors
Hongbo Gao, Zhengyu Li, Xueru Nie, Dihao Zhu, Lijun Zhao, Yunke Wang, Chang Xu
Abstract
RGB-Thermal (RGB-T) salient object detection leverages complementary cues from visible and thermal modalities to improve robustness in challenging environments. However, in real-world scenarios, the reliability of each modality is inherently unstable: RGB images degrade under low illumination, motion blur, and noise, while thermal imagery often suffers from contrast compression and sensor artifacts. Such degradation introduces unreliable perceptual evidence that can mislead cross-modal fusion and significantly deteriorate detection performance. To address this challenge, we propose RA-SOD, a reliability-aware RGB-T salient object detection framework that explicitly models modality reliability and integrates it into feature learning and cross-modal fusion. First, we introduce a reliability-conditioned representation that adaptively compensates degraded modality features while preserving structural cues. Second, an uncertainty-guided dual-stream refinement strategy progressively corrects cross-modal representations while suppressing unreliable evidence. Finally, we propose a pixel-wise modality competition mechanism that dynamically selects modality cues according to spatial reliability for fine-grained fusion. Extensive experiments on four benchmarks (VT821, VT1000, VT5000, and VT-IMAG) demonstrate that RA-SOD achieves state-of-the-art performance and exhibits strong robustness under severe modality degradation. Code and models are available at https://github.com/zaoxienian/RA-SOD.