Neural radiance fields improve sparse view 3D photo quality
DriftNeRF: Cross-Ray Density Drifting for Sparse-View Neural Radiance Fields
Computer Vision and Pattern RecognitionMachine Learning
Summary
Neural Radiance Fields (NeRF) create 3D views from many images but struggle when only a few images are available. The authors found that the usual way of training NeRF lets different camera rays assign conflicting densities to the same space, causing inconsistencies. They created DriftNeRF, a method that encourages agreement between these rays by sharing information about 3D densities, improving quality especially when images are sparse. DriftNeRF works without extra data or pre-training and shows better accuracy on several benchmarks.
What this means in practice
- •For computer vision engineers: Enhance sparse-view 3D scene reconstruction quality without additional images or pre-trained models using DriftNeRF's cross-ray density regularization.
- •For graphics software developers: Integrate a drop-in regularizer to improve photorealistic novel view synthesis from limited images in NeRF pipelines without relying on geometry or depth preprocessing.
Authors
Juntang Wang, Yihan Wang, Sheng'en Li, Hao Wu, Dongmian Zou, Shixin Xu
Abstract
Neural Radiance Fields (NeRF) synthesize photorealistic novel views from dense image collections, but degrade sharply when few views are available. We attribute a specific, measurable defect to the objective itself: the rendering loss is a sum over independent rays and contains no term comparing densities that different rays assign to the same region of space, so nearby 3D points can settle on contradictory densities while every training pixel is reproduced correctly. We make this quantity explicit by measuring cross-ray density disagreement directly during training, and show that it stays high in vanilla NeRF, falls only incidentally under frequency-based regularization, and moves together with held-out quality in the sparse regime. We then close the gap with DriftNeRF, a drop-in regularizer that instantiates the drifting-field template on density samples: a masked exponential kernel aggregates densities from nearby points on other rays into a stop-gradient consensus target, and the resulting drifting field penalizes unsupported disagreement while leaving agreed-upon boundaries intact. The regularizer reads only sampled positions, ray identifiers, and predicted densities, so it requires no pretrained model, external geometry, or additional images, and transfers across field parameterizations. On NeRF-Synthetic it reduces residual disagreement by an order of magnitude, and across NeRF-Synthetic, DTU, and LLFF it improves 3-view PSNR over the strongest MLP-NeRF baseline by 0.42-0.65 dB and remains competitive with sparse-view Gaussian pipelines under a matched protocol without their depth, flow, or covisibility preprocessing.