Neural residual fields improve 3D Gaussian splatting for better visuals
NRF-GS: Neural Residual Fields for Expressive and Compact Gaussian Splatting
Computer Vision and Pattern RecognitionGraphics
Summary
Rendering detailed 3D scenes efficiently is hard because simple color models miss shiny reflections, so they use many small shapes, or splats, to fix that. The authors found that instead of using basic methods to represent how light reflects, combining a simple base with a shared neural network that adds complex light effects improves quality. This method, called NRF-GS, lets scenes use fewer splats while showing clearer shiny and detailed surfaces. As a result, it makes 3D scene rendering both better and more compact.
What this means in practice
- •For 3d graphics developers: Render detailed 3D scenes with fewer splats to achieve faster and higher-quality visuals by integrating neural residual networks into Gaussian splatting.
- •For visual effects studios: Create more realistic lighting and shiny surface effects in 3D scenes using a hybrid representation to reduce rendering complexity without sacrificing detail.
Authors
Pratik Singh Bisht, Andreas Kolb
Abstract
We revisit the role of appearance modeling in 3D Gaussian Splatting (3DGS) and show that limited expressiveness in view-dependent reflectance is a key driver of representation redundancy. In standard 3DGS, low-order spherical harmonics (SH) are used, restricting the splats' ability to model high-frequency directional effects, which is typically compensated by increasing the number of splats. We propose \emph{NRF-GS: Neural Residual Fields for Gaussian Splatting}, a hybrid representation that replaces per-splat SH-bases with a shared neural residual field. Each Gaussian encodes a compact set of appearance features and a lambertian base color, while a lightweight \emph{global scene-level MLP} predicts view-dependent residuals conditioned on viewing direction, distance, and per-splat features. This formulation enhances directional reflectance modeling by combining diffuse per-splat reflectance representations with a shared global function for high-frequency details, enabling both higher expressiveness and parameter sharing across splats. Our key insight is that by accurately capturing high-frequency directional reflectance, especially in specular regions, the GS-representation becomes more expressive, reducing the need for geometrically redundant splats. As a result, NRF-GS achieves comparable or better rendering quality while reducing the number of Gaussians by up to 50\%, and produces visibly improved specular and high-frequency details.