Gaussian splatting improves reflective surface rendering with geometry learning
RGS: Reflection-aware Gaussian Splatting via Learning Geometry Continuity for Reflective Objects
Computer Vision and Pattern Recognition
Summary
Rendering shiny or reflective objects in 3D often leads to problems with their shape and surface details. The authors found that existing methods sometimes produce inaccurate or collapsed surfaces in these shiny areas. They developed a new technique called Reflection-aware Gaussian Splatting (RGS) which uses a 3D model to help maintain the correct shape and better capture reflections from different angles. Their method creates smoother surfaces and more realistic reflections, leading to higher quality images of shiny objects from new viewpoints.
What this means in practice
- •For computer graphics developers: Enhance 3D rendering engines to produce more accurate and visually pleasing reflections on shiny objects in virtual scenes.
- •For augmented reality app developers: Generate realistic views of reflective objects from new angles to improve user immersion in AR experiences.
Authors
Xiaobiao Du, Yida Wang, Cheng Bi, Kun Zhan, Xin Yu
Abstract
Gaussian Splatting has significantly improved the quality of novel view synthesis with explicit Gaussian representation. However, we observed that existing 3D Gaussian Splatting methods (3DGS) often suffer from surface collapse issues on reflective regions, and thus produce inferior geometry and low-quality specular. In this work, we propose a physically-based deferred rendering framework, named Reflection-aware Gaussian Splatting (RGS), that can accurately model specular regions and improve novel view synthesis performance. Specifically, we found that a powerful 3D foundation model can provide a strong 3D geometric prior to foster correct geometric modeling. Based on this, we propose a cross-view shape consistency regularization to regularize the geometry surface with the large model prior and cross-view constraints. In this manner, our RGS can produce smoother geometric surfaces on reflective regions while reducing geometric hollows. To further improve rendering results on reflective regions, we present a reflection-aware densification strategy that is designed to capture specular variations across various views. With this strategy, our RGS is able to render novel views of objects in higher quality. Extensive experiments demonstrate our method consistently renders high-quality reflective objects, achieving state-of-the-art performance.