Improving 3D scene geometry from few images using tensor voting
TV-SGS: Gaussian Splatting with Geometric Information Propagation via Tensor Voting under sparse views
Computer Vision and Pattern Recognition
Summary
Creating 3D scenes from photos is tricky, especially when you only have a few pictures. The authors improved a method called Gaussian Splatting by making parts of the 3D model communicate directly using a technique called Tensor Voting. This helps the 3D shapes become clearer and more accurate, even when there aren’t many pictures to guide the process. Their approach works well with existing methods and improves how 3D scenes look and feel.
What this means in practice
- •For 3d graphics developers: Improve geometric accuracy in 3D scenes generated from limited camera views during virtual environment creation or augmented reality.
- •For robot perception teams: Enhance 3D scene understanding for robots using sparse sensor data by improving geometric structure reconstruction.
Authors
Harish N Sathishchandra, Philippos Mordohai
Abstract
Gaussian Splatting has been effective in inferring scene representations that excel in novel view synthesis. Multiple splats cooperate seamlessly to synthesize the pixels of novel views and are jointly optimized even though they only affect each other indirectly, via pixels they project to in common. We present an approach that enables direct communication among splats to enhance the geometric structures they form in 3D. This is accomplished by Tensor Voting, which was originally designed to infer structures from noisy inputs and has been adapted here to provide supervision during test-time optimization, leading to more accurate scene geometry. We introduce a new class of 3D losses that do not rely on rendering and can be combined with essentially all losses previously reported in the literature. Our 3D losses are especially effective when the input views are sparse and geometric regularization is essential due to limited supervision from the images. Our method is easy to integrate with a diverse set of backbones, and our experiments on the DTU and Tanks-and-Temples datasets demonstrate that TV-SGS improves the geometry of the outputs compared to the backbone, while maintaining or improving rendering quality.