Visibility splatting boosts 3D rendering with better light and material separation
VisibilitySplat: Ray-Traced Occlusion for Relighting with 3D Gaussian Splatting
Computer Vision and Pattern RecognitionGraphics
Summary
Creating realistic 3D pictures often requires understanding how light and materials interact. The authors improve a way to build 3D scenes called 3D Gaussian Splatting by adding a smart method that checks how objects block or let light pass. This helps computers figure out how things should look under different lighting and from new viewpoints. Their method runs efficiently and still works well with existing tools, showing better results in tests than some earlier techniques.
What this means in practice
- •For game developers: Produce realistic lighting in interactive 3D scenes using efficient visibility checks that improve material and light separation.
- •For visual effects artists: Create more accurate relighting of 3D models for film and animation by integrating sparse ray-traced visibility with existing rendering pipelines.
Authors
Sven Kluge, Oliver Staadt
Abstract
We address inverse rendering with 3D Gaussian Splatting (3DGS) by introducing explicit visibility modeling for relighting as well as material and light decomposition. Our approach augments the standard rasterized 3DGS pipeline with a sparse visibility oracle that estimates transmittance-weighted occlusion for Gaussian primitives from sampled lighting directions. This oracle is implemented using a point-based ray tracer accelerated by a bounding volume hierarchy (BVH) and queried sparsely within the inverse-rendering objectives, while the original 3DGS loop is retained for training efficiency, interactive visualization, and tooling compatibility. Shading follows a physically based rendering (PBR) formulation with a principled microfacet bidirectional reflectance distribution function (BRDF) and low-resolution high-dynamic-range (HDR) environment maps. We evaluate our method on the TensoIR synthetic dataset against baselines based on neural radiance fields (NeRFs), tensor representations, and 3DGS. The results show competitive performance and improvements over several earlier baselines for novel-view synthesis (NVS), albedo reconstruction, and relighting. Ablations indicate that explicit visibility modeling is an important contributor to relighting and material-illumination disentanglement, while pure PBR supervision can favor appearance fitting in NVS. Overall, the findings suggest that sparse, point-based visibility queries effectively complement rasterized splatting for inverse rendering while preserving the practical advantages of the 3DGS ecosystem.