Tensor voting improves 3D shapes from few images using gaussian splats
TV-SGS: Gaussian Splatting with Geometric Information Propagation via Tensor Voting under sparse views
Computer Vision and Pattern Recognition
Summary
Making 3D pictures from a few photos is hard because there isn’t much information to work with. The authors developed a method that helps small 3D pieces, called splats, talk directly to each other to build better shapes. They use a technique called tensor voting, which was originally made to find structures even in noisy data. This helps improve the overall geometry without needing extra images. Their method works well with existing tools and makes the 3D scenes look better and more accurate.
Gaussian splatting3D scene representationnovel view synthesistensor votinggeometric regularizationtest-time optimizationsparse views3D loss functionsDTU datasetTanks-and-Temples dataset
Authors
Harish N Sathishchandra, Philippos Mordohai
Abstract
Gaussian Splatting has been effective in inferring scene representations that excel in novel view synthesis. Multiple splats cooperate seamlessly to synthesize the pixels of novel views and are jointly optimized even though they only affect each other indirectly, via pixels they project to in common. We present an approach that enables direct communication among splats to enhance the geometric structures they form in 3D. This is accomplished by Tensor Voting, which was originally designed to infer structures from noisy inputs and has been adapted here to provide supervision during test-time optimization, leading to more accurate scene geometry. We introduce a new class of 3D losses that do not rely on rendering and can be combined with essentially all losses previously reported in the literature. Our 3D losses are especially effective when the input views are sparse and geometric regularization is essential due to limited supervision from the images. Our method is easy to integrate with a diverse set of backbones, and our experiments on the DTU and Tanks-and-Temples datasets demonstrate that TV-SGS improves the geometry of the outputs compared to the backbone, while maintaining or improving rendering quality.