Hierarchical octree improves 3D Gaussian splatting for view and depth synthesis
OctreeSplat: Hierarchical Octree-Guided Feed-Forward Gaussian Splatting
Computer Vision and Pattern Recognition
Summary
Creating realistic 3D views and reconstructions from images can be slow and complicated because detailed areas need more computing resources. The authors present OctreeSplat, a method that organizes 3D data in a flexible, layered way called an octree, so areas that are more complex get more attention. This approach uses information from existing visual models to guide where to add details and improves how well the 3D shapes and depths are reconstructed. It does this efficiently without needing long scene-by-scene adjustments and works well on several challenging datasets.
What this means in practice
- •For computer graphics developers: Generate detailed 3D scenes for gaming and virtual reality with fewer computing resources by adapting details to scene complexity.
- •For robotics teams: Improve robot perception by providing more accurate 3D depth and surface models from multi-view images without slow per-scene training.
Authors
Yibin Zhao, Yihan Pan, Wenli Yang, Jun Nan, Jianjun Yi
Abstract
3D Gaussian Splatting (3DGS) has shown considerable potential for novel view synthesis (NVS) and 3D reconstruction. Feed-forward methods avoid costly per-scene optimization, but fixed image or spatial resolutions limit their ability to allocate primitives according to local complexity. We present OctreeSplat, a hierarchical octree-guided feed-forward Gaussian splatting. OctreeSplat extracts multi-view features and geometric priors from a frozen visual foundation model (VFM), constructs a hierarchical octree, and predicts densification scores. The resulting representation allocates Gaussian primitives at spatially varying densities according to local complexity. Stochastic solids volume rendering improves depth rendering, strengthens geometric supervision, and supports surface reconstruction. When camera parameters are available, an optional test-time optimization (TTO) module improves agreement between predicted and ground-truth parameters. Training updates only 76 million additional parameters and requires 35 GPU-hours. Experiments on Replica, TUM, Tanks and Temples, and DTU demonstrate competitive RGB view synthesis, depth synthesis, and surface reconstruction. OctreeSplat reduces the number of Gaussian primitives while maintaining high reconstruction quality.