Geometry regularization improves depth accuracy in neural radiance fields
Geometry Regularization for Vanilla NeRF via Self-Supervised Cross-Ray Consistency
Computer Vision and Pattern RecognitionGraphics
Summary
Neural Radiance Fields (NeRF) create 3D scenes from images but often have noisy or unclear depth details. The authors introduce a way to make the 3D shapes clearer by checking color consistency from different viewpoints without needing extra depth data. They do this by creating virtual rays that meet at guessed surface points and making sure the colors match well, which helps refine the shape. This method made the 3D shapes more accurate by over 30% in their tests.
What this means in practice
- •For 3d graphics developers: Enhance 3D scene reconstruction quality by improving geometric accuracy without extra depth sensors.
- •For augmented reality engineers: Create more reliable virtual object placements with better scene geometry from only images.
Authors
Zongji Wang
Abstract
Neural Radiance Fields (NeRF) achieve high-quality novel-view synthesis by representing a scene as a continuous volumetric radiance field. However, the geometry encoded by the density field of vanilla NeRF can be noisy and inaccurate, since different combinations of shape and radiance may explain the same input images. To reduce this ambiguity, we augment vanilla NeRF with a geometry-optimization branch based on self-supervised cross-ray consistency. Specifically, this branch samples candidate surface points from the density field, generates virtual rays that intersect at each point from different directions, and renders their appearance using the NeRF network. Based on the visibility-weighted color consistency across these rays, we constrain the implicit geometry. We further regularize the implicit geometry by penalizing opacity at candidates with inconsistent appearance predictions and encouraging local normal smoothness. These geometric constraints are derived from the NeRF predictions and require no ground-truth depth or surface-normal supervision. Experiments on eight Replica scenes show that the proposed method reduces the mean Chamfer distance by 30.8% compared with vanilla NeRF.