Continuous neural color fields improve 3d gaussian splatting quality
3D Gaussian Splatting within Self-Learned Neural View-Dependent Color Fields
Computer Vision and Pattern Recognition
Summary
Generating 3D images from new viewpoints is tricky because existing methods using 3D Gaussian shapes often create blurry or floating artifacts and miss fine details. The authors propose using a continuous color model powered by neural networks to better capture subtle color changes that relate smoothly to geometry. They also add a mathematical constraint to keep color and position changes aligned. Their approach learns these details without extra data, resulting in clearer and more accurate 3D scenes.
What this means in practice
- •For 3d graphics developers: Enhance rendering engines to produce more detailed and artifact-free 3D views from limited input images using improved neural color fields.
- •For augmented reality app makers: Create more realistic and smoothly rendered virtual objects and scenes that adapt well to new viewpoints without extra scene information.
Authors
Yixiao Yang, Yu-Shen Liu, Zhizhong Han
Abstract
3D Gaussian Splatting (3DGS) has become a vital tool in novel view synthesis. 3D Gaussian functions with attributes like color, opacity, and scale are stacked together to explicitly model a radiance field. However, discrete 3D Gaussians usually produce poor depth with few geometry details but lots of floaters, which hinders the effective modeling of smooth radiance fields. To address this issue, we propose to learn a radiance field within a continuous view-dependent color field parameterized by neural networks. With continuous constraints in color, the bias on low-frequency signals of neural networks can highly encourage the geometry modeling to contribute to high-frequency color variations on images, getting rid of solely relying on the color attribute itself. Moreover, we impose a constraint to improve the Lipschitz continuity of the neural color field, making large changes of each Gaussian's color also match with large changes of its position in terms of a ratio. To this end, we additionally introduce novel self-learning strategies to improve the learning of the neural view-dependent color field without any external knowledge or priors, aiming for better generalization on unseen scenes. Our numerical and visual comparisons on widely used benchmarks justify our idea and show better ability of high fidelity geometry recovery in 3DGS. Project page: https://github.com/yangyixi21/Neural-Color-Fields-3DGS.