Stable 3D scene learning improves without known camera poses
SG-Splat: Stable Geometry Learning under Pose Uncertainty for Pose-Free 3D Gaussian Splatting
Computer Vision and Pattern Recognition
Summary
Creating 3D scenes from images usually needs exact camera positions, but this paper tackles uncertainties in those positions. The authors present a system called SG-Splat that separates shape and color learning to reduce errors caused by pose mistakes. They also use a two-step process to better align images and improve 3D points before coloring them. This approach makes the 3D reconstruction more stable and accurate, even when camera positions are not perfectly known.
What this means in practice
- •For augmented reality developers: Create realistic 3D scenes from casual photos without needing exact camera positions, improving AR content generation robustness.
- •For 3d mapping teams: Generate stable 3D reconstructions from unposed video footage, reducing errors from unknown or imprecise camera pose data.
Authors
Xiaohan Lu, Jiaqi Zhang, Weijia Jiang, Siwei Ma
Abstract
3D Gaussian Splatting (3DGS) has become a promising representation for real-time novel view synthesis and 3D scene reconstruction. For practical deployment, recent studies have explored pose-free feed-forward 3DGS, which reconstructs Gaussian representations directly from unposed input images. However, these methods remain sensitive to residual pose errors, which perturb canonical-space Gaussian transformation, introducing noisy supervision for geometry and attribute learning. To address this problem, we present SG-Splat, a pose-free feed-forward 3DGS framework that improves robustness to pose uncertainty and stabilizes the learning of Gaussian primitives. SG-Splat decouples geometry estimation and Gaussian attribute prediction into two dedicated branches, reducing error coupling between pose-sensitive geometry and Gaussian attributes. We further introduce a coarse-to-fine refinement pipeline, consisting of Flow-Guided Pose Refinement (FGPR) and Residual-Guided Geometry Correction (RGC), to improve geometric alignment before Gaussian attribute reconstruction. Specifically, FGPR refines camera poses with flow-based correlation features and differentiable bundle adjustment. RGC then corrects initial point maps using depth residuals derived from FGPR, producing more reliable Gaussian centers. To stabilize training, we introduce a dual-path strategy that mixes ground-truth and predicted poses for canonical transformation. It provides clean supervision for Gaussian attributes while improving geometry robustness to residual pose errors. Extensive experiments on DL3DV and RealEstate10K show that SG-Splat achieves state-of-the-art performance in both novel view synthesis and pose estimation.