Gaussian splatting improves rendering of reflections in 3D scenes

Gaussian Splatting with Reflections Guided by What is Seen

Computer Vision and Pattern Recognition

Summary

It can be hard to create realistic images of shiny surfaces or reflections when making 3D scenes from photos. The authors show a new way to use a type of image processing called Gaussian splatting that pays attention to reflections you can actually see in the pictures, rather than just guessed lighting. This helps make the reflections and shiny parts look more accurate and realistic in new views of the scene. Their tests show their method works better than older approaches, especially for tough near reflections close to objects.

What this means in practice

  • For visual effects teams: Produce more accurate shiny and reflective effects for 3D scenes derived from photographs, improving visual realism in films and games.
  • For augmented reality developers: Create augmented reality visuals with better near-field reflections that match real-world environments seen in input images.

Authors

Yiming Liang, Tianyu Xiao, Hiroshi Ishikawa

Abstract

While novel view synthesis methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have shown impressive performance on common scenes, they often struggle with capturing view-dependent specular effects, particularly near-field reflections. Existing methods utilize environment maps, which only represent lights from infinity, or environment Gaussians, which rely solely on indirect supervision from reflections, are insufficient for accurately modeling such effects. Here, we propose a framework based on 2D Gaussian Splatting (2DGS) that leverages directly observable environment information in input images. Our approach also integrates a hybrid reflection appearance representation and view-dependent specular modeling. Experiments on public benchmarks and a newly curated near-field reflection dataset demonstrate that our method achieves superior rendering quality and geometric accuracy.