Geometry aware gaussian splatting improves 3D surface reconstruction quality
PCAsplat: Gaussian Splatting with Local PCA Regularization
Computer Vision and Pattern RecognitionGraphics
Summary
3D images are created by placing little blobs called Gaussians, but some blobs can float in empty space because they don't get enough feedback when hidden or unseen. The authors developed PCAsplat, which uses a smart math technique called PCA to gently steer these blobs to fit the real surface better, even when they are not directly seen. This makes the 3D shapes cleaner and more accurate, reducing floating blobs and improving tasks that work with these shapes. Their method works well on several standard tests and still produces good new views and mesh shapes.
What this means in practice
- •For 3d graphics developers: Produce cleaner 3D reconstructions from images by reducing floating artifacts and improving surface alignment of Gaussian blobs.
- •For geometry processing engineers: Use surface-aligned Gaussian representations as a better starting point for point cloud segmentation and Poisson mesh reconstruction.
Authors
Vitor Matias, Filipe Nascimento, Kiyohiro Nakayama, João Paulo Lima, Márcus Lobo, Gordon Wetzstein, Leonidas Guibas, Afonso Paiva, Tiago Novello
Abstract
Gaussian splatting has emerged as a flexible representation for 3D reconstruction from posed images. However, existing methods are optimized primarily using rasterization-based losses, which supervise a splat only when it contributes to sampled camera rays. Gaussians that are occluded or contribute little to the sampled view therefore receive weak or no geometric gradients and may drift away from the underlying surface, producing undesired floaters. We introduce PCAsplat, a geometry-aware regularization framework for Gaussian splatting based on differentiable local principal component analysis (PCA). Our PCA regularizer acts directly on neighborhoods of Gaussian centers and can therefore update Gaussians that do not contribute to the current training view. We regularize the PCA eigenvalues to encourage Gaussians to move to the underlying surface with isotropic tangent-plane coverage. We also align each Gaussian normal with the PCA-estimated neighborhood normal to enforce consistent orientation. Experiments on DTU, Tanks and Temples, and NeRF Synthetic show that the splats produced by PCAsplat better approximate samples of the reference surface while substantially reducing undesired floaters. These surface-aligned splats enable downstream geometry-processing tasks, including point cloud segmentation, and direct Poisson reconstruction. Additionally, PCAsplat remains competitive under conventional novel view synthesis and mesh extraction tasks. Code will be released.