Voxel selective splatting improves 3d object reconstruction from few images
VS-Splat: Voxel-Selective feed-forward Gaussian Splatting for end-to-end 3D object reconstruction from sparse-views
Computer Vision and Pattern Recognition
Summary
Reconstructing 3D objects from just a few photos is hard because many 3D models waste effort representing empty space instead of the object. The authors came up with a new method called VS-Splat that focuses on likely parts of the 3D space where the object is, using only 2D images to guide the process. This approach places fewer points outside the object and captures finer details better. Their tests show VS-Splat works better than previous methods, and it can also improve other 3D reconstruction techniques and handle camera errors better.
What this means in practice
- •For 3d graphics developers: Build tools that reconstruct detailed 3D objects from just a few photographs with improved accuracy and efficiency.
- •For augmented reality engineers: Enhance AR experiences by creating more accurate and detailed 3D object models using sparse camera views without extra 3D data.
Authors
Yunsu Jeong, Hyuk Heo, Youngsang Kwak, Jaehwa Kwak, Il Yong Chun
Abstract
Feed-forward Gaussian splatting models have demonstrated remarkable effectiveness in reconstructing three-dimensional (3D) objects from a few two-dimensional (2D) images, even if they are unseen. As existing methods typically predict Gaussian primitives uniformly across the 3D space, most primitives are placed in non-object regions. This may hinder the representation of fine object details. This paper proposes a Voxel-Selective Gaussian Splatting model (VS-Splat), a new end-to-endfeed-forward Gaussian splatting framework that predicts many primitives only within selected voxels that are likely to belong to an object, without 3D structural supervision. To achieve this, we propose a new learnable voxel selection approach that identifies object-centric voxels only with 2D rendering supervision. Our sparse-view rendering experiments with three benchmark datasets show that proposed VS-Splat outperforms several state-of-the-art methods. We further demonstrate its effectiveness as a backbone for an existing densification method and show that anoptional extension improves its robustness to inaccurate camera pose estimates.