New method improves 3D scene reconstruction under fixed budgets
Efficient Confidence-Aware Densification for 3DGS
Computer Vision and Pattern RecognitionGraphics
Summary
Creating detailed 3D models using limited resources is challenging because some methods focus too much on well-reconstructed parts and neglect others. The authors propose a way to add new 3D points based on where the image errors are largest and where the system is least confident about the reconstruction. This approach helps put more effort into improving less accurate areas, resulting in better overall 3D models. Their method works well without needing any prior information and improves quality across different resource levels.
What this means in practice
- •For 3d graphics engineers: Improve quality of 3D reconstructions from images under fixed computational budgets by focusing resources on uncertain and error-prone regions.
- •For augmented reality developers: Generate more accurate 3D scene representations in AR applications without needing pre-trained models or dense initializations, optimizing resource use.
Authors
Manuel Hofer, Thomas Köhler, Felix Windisch, Bernhard Kerbl, Markus Steinberger, Lukas Radl
Abstract
3DGS-MCMC has replaced the gradient-driven density control of 3D Gaussian Splatting with opacity-based sampling and relocation, and is now the de facto standard for prior-free densification under a fixed primitive budget. Because opacity stays high where the scene is already well reconstructed, this sampling sends most new primitives to the foreground and leaves the background under-reconstructed. At low budgets, prior-based methods such as ConeGS and EDGS outperform it, at the cost of a pre-trained radiance field or dense image matches and a separate initialization stage. We argue for a comeback of error-driven densification within the fixed-budget MCMC framework. Our confidence-aware densification keeps the fixed budget and gradual growth of 3DGS-MCMC, but places primitives by the image error instead of the opacity. New primitives are drawn mostly where screen-space gradients are large. We favor primitives with a low learned confidence, which marks regions the reconstruction still explains poorly, and relocate the primitives that contribute least to the rendered images to these places. Averaged over 24 scenes, our method outperforms 3DGS-MCMC in PSNR, SSIM and LPIPS at every budget from 100k to 2M primitives. At 100k, without a prior or an initialization stage, it recovers 0.84dB of the 1.05dB PSNR by which ConeGS leads 3DGS-MCMC, and from 500k on, it has the highest PSNR and SSIM of all fixed-budget methods we compare with, including ConeGS and EDGS. While inherently prior-free, our densification can also exploit priors: from the ConeGS or EDGS initialization, it matches ConeGS in PSNR and SSIM at 100k and is best in all three metrics at 500k.