DerainSplat: Feed-Forward Clean 3D Gaussian Splatting from Sparse Rainy Views
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the challenge of cleaning up 3D scenes created from a few rainy images, which is important for technologies like self-driving cars. They developed DerainSplat, a system that removes rain effects in a single step and rebuilds a clear 3D scene. To train their method, they created a large dataset simulating different rain conditions and designed a network to recognize and correct weather artifacts. Their approach improves depth and appearance accuracy by using special support maps and cycle consistency. Experiments show that their system works better than existing methods on multiple datasets and real rainy scenes.
image deraining3D Gaussian Splattingmulti-view reconstructionweather factor synthesiscost-volume matchingappearance fusiondepth estimationcycle consistencycomputer visionembodied AI
Authors
Fuzhen Jiang, Changyue Shi, Chuxiao Yang, Xinyuan Hu, Wenjie Ye, Minghao Chen
Abstract
Although image deraining has advanced substantially, existing methods mainly focus on 2D image restoration. As spatial intelligence applications such as embodied AI and autonomous driving continue to emerge, reconstructing clean 3D scenes from sparse rainy views in a feed-forward manner becomes increasingly important. Existing feed-forward 3D Gaussian Splatting (3DGS) methods often assume clean inputs and collapse under rainy conditions. To this end, we present \textbf{\textit{DerainSplat}}, a feed-forward framework that reconstructs clean 3D scenes from only a few rainy views. To support this task, we build a large-scale multi-view derain dataset through a four-stage synthesis pipeline that sequentially models overcast illumination, depth-dependent haze, rain streaks, and lens raindrops, producing privileged weather factors. We introduce a weather net that predicts the weather factors from rainy context and yields two support maps. Scene support modulates cross-view cost-volume matching, while radiance support drives depth-aligned appearance fusion to fill corrupted pixels. The derived geometry evidence further attenuates Gaussian opacity to reduce spurious structures. A rainy cycle consistency re-renders clean views using the predicted factors and aligns them with rainy inputs. Extensive experiments show that \textbf{\textit{DerainSplat}} outperforms existing methods on various datasets, including RealEstate10K, ACID, Mip-NeRF360, and real-world rainy scenes, with strong cross-dataset generalization.