Semantic guided 3D gaussian splatting improves urban scene reconstruction
3D Gaussian Splatting for Fine-grained Urban Scene Reconstruction via Semantic-guided Density Control and Loss Design
Computer Vision and Pattern Recognition
Summary
Building detailed 3D models of cities is hard because usual methods don't handle tiny structures well and miss important boundaries. The authors improve a technique called 3D Gaussian Splatting by using AI to identify different parts of a scene and guide the reconstruction process. This helps focus on critical areas like edges and lines, producing clearer and more accurate models of urban environments. Their tests show better quality and fewer errors compared to previous methods.
What this means in practice
- •For urban planning teams: Generate higher precision 3D city models that capture fine architectural details to support planning and visualization tasks.
- •For game developers: Create realistic urban environments with fewer visual artifacts for use in interactive simulations and games.
Authors
Yueling Liao, San Jiang, Xing Zhang, Zhongwen Hu
Abstract
3D Gaussian Splatting (3DGS) has been becoming a promising 3D reconstruction technique because of its advantage of high-fidelity rendering effect with high efficiency. However, it still suffers from coarse density control and semantically blind loss functions when reconstructing the fine-grained structures in complex urban scenes. Thus, this study proposes SG-GS, a semantic-guided 3DGS method for fine-grained structure reconstruction. It takes the semantic segmentation deduced by the Segment Anything Model (SAM) as a constraint and systematically integrates it into both Gaussian density control and loss function design. On the one hand, a semantic-constrained adaptive density control mechanism is constructed, which dynamically allocates gradient thresholds to different semantic regions of the scene and introduces an additional boundary-enhanced densification strategy, so that new Gaussians are directed to structurally critical regions. On the other hand, a multi-feature guided loss function is designed, and fine-grained supervision signals are gradually introduced through a phased progressive training strategy, alleviating the artifacts caused by the lack of semantic constraints in the basic loss. Experiments on three datasets show that the proposed method consistently improves reconstruction accuracy. Compared with the original 3DGS, the PSNR increases by approximately 2.7% to 5.2%, while artifacts on linear structures and boundary blurring are effectively mitigated. The method can serve the fine-grained 3D reconstruction of urban scenes.