RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weather
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the difficulty of detecting 3D objects in bad weather, where sensors like LiDAR and RADAR struggle due to fewer signal returns. They use cameras to help, but cameras also get blurry or blocked by rain or snow. Their new method, called Reliability-Aware Fusion (RAF), teaches the system to know which camera parts are trustworthy and which are not, improving the overall detection. They tested their approach on real datasets and found it makes detection more accurate compared to methods using only LiDAR and RADAR.
3D object detectionLiDAR4D RADARcamera fusionadverse weatherpoint cloudconfidence mapsper-pixel reliabilityBird’s Eye View (BEV)detection accuracy
Authors
Heejun Park, Jaeseok Jeong, Kuk-Jin Yoon
Abstract
Robust 3D object detection in adverse weather conditions is challenging due to sensor limitations. Although combining complementary modalities such as LiDAR and 4D RADAR has shown promise, the sparsity of these sensors becomes apparent in adverse weather with reduced reflections, leading to objects with few or no point cloud returns. To address this limitation, camera sensors provide visual cues even when LiDAR and RADAR signals are weakened. However, cameras themselves are also vulnerable to adverse weather, where some regions become unreliable due to snow or rain occluding the camera lens. While some camera-fusion methods designed for adverse weather learn to weigh image regions via confidence maps, these maps receive no direct supervision and are learned solely through the detection loss. We introduce Reliability-Aware Fusion (RAF), which explicitly supervises per-pixel reliability estimation and provides a direct learning signal for identifying and suppressing unreliable visual cues. Our framework leverages pretrained LiDAR-RADAR networks, keeping their backbones frozen while only training the added camera branch, BEV fusion encoder, and detection head. Extensive experiments on the K-Radar and VoD datasets demonstrate that integrating RAF consistently improves detection accuracy over LiDAR-RADAR baselines, achieving up to +6.5 $AP_{BEV}$ and +7.4 $AP_{3D}$ gains. Code is available at https://github.com/parkie0517/RAF.