DecomVoxel improves 3D scene reconstruction with better object completion
DecomVoxel: Harnessing 3D-Native Priors with Guided In-situ Denoising Optimization for Decompositional Scene Reconstruction
Computer Vision and Pattern Recognition
Summary
Reconstructing objects in 3D spaces is hard, especially when they block each other. The authors propose DecomVoxel, a new method that reduces errors by combining 3D-aware guidance with a smart cleanup step. Their method keeps the scene's true shape and texture more accurate than previous ones. This results in clearer and more realistic 3D reconstructions of complex environments.
What this means in practice
- •For virtual reality developers: Create more accurate and detailed 3D models of complex spaces to enhance immersion and interaction in VR experiences.
- •For robotics engineers: Improve robots’ understanding of cluttered real-world environments by reconstructing objects and backgrounds more reliably under occlusions.
Authors
Junfeng Ni, Zirui Zhou, Yixin Chen, Yu Liu, Nan Jiang, Zhifei Yang, Song-Chun Zhu, Siyuan Huang
Abstract
Decompositional scene reconstruction aims to reconstruct high-quality objects and background, yet existing methods still struggle with the level of quality under heavy occlusions. While generative priors offer a potential solution, 2D image-based priors often suffer from multi-view inconsistency due to a lack of 3D awareness. Conversely, 3D-native priors provide stronger structural inductive biases but frequently lead to spatial drift and misalignment within complex scenes. To address these issues, we propose DecomVoxel, formulating object completion as a guided in-situ denoising optimization that bridges 3D-native priors with neural scene reconstruction. Our framework introduces a reformulated epsilon-based distillation loss to ensure stable latent refinement, alongside adaptive spatial guidance that utilizes occupied and vacant anchors with temporal annealing to suppress generative hallucinations and mitigate spatial drift. Experiments on Replica and ScanNet++ show that DecomVoxel significantly outperforms state-of-the-art methods while faithfully preserving the original spatial layout, structural fidelity, and style-consistent texture. Our method pushes the boundary of decompositional reconstruction by delivering high-quality textured meshes with clean topology, geometry, and appearance, providing a robust solution for the decompositional reconstruction of complex real-world scenes. Code is available at https://github.com/DecomVoxel/DecomVoxel.