Graph3R improves 3D scene rebuilding from mixed image sets fast
Graph3R: Unifying View Graph Initialization with Feed-Forward Reconstruction
Computer Vision and Pattern RecognitionRobotics
Summary
Rebuilding 3D scenes from photos usually assumes all pictures come from one place, but real collections often have mixed scenes. The authors present Graph3R, which groups photos by scene automatically and reconstructs each one separately using shared features. This method combines figuring out which photos belong together with rebuilding the scene, making the process both more accurate and faster. It works well on tests and also speeds up repeated mapping tasks significantly.
What this means in practice
- •For photogrammetry teams: Automatically separate mixed collections of photos by scene and reconstruct each scene accurately and efficiently.
- •For robotics mapping engineers: Speed up and improve multi-session localization and mapping by grouping views and reconstructing scenes in a unified process.
Authors
Jingwen Yu, Hengli Wang, Jianhao Jiao, Zhonghang Liu, Ping Tan, Hong Zhang
Abstract
Feed-forward reconstruction recovers camera poses and dense geometry from image sets whose views are assumed to come from a single scene. In practice, uncurated image collections often contain views from disjoint scenes, and reconstructing these views jointly can corrupt camera poses and geometry. Existing pipelines typically resolve view associations in separate stages before final reconstruction, leaving view graph initialization and reconstruction inference only loosely coupled. We introduce Graph3R, which unifies view graph initialization with feed-forward reconstruction through a shared geometric representation. A lightweight verification head predicts pairwise covisibility from the multi-view features of $π^3$ to construct a view graph. This graph classifies the input views into scene groups. For each group, we mask the shared features before decoding camera poses and dense geometry. Reusing the multi-view features allows Graph3R to reconstruct disjoint scenes separately with a single backbone pass. Across three benchmarks, Graph3R improves average maximum recall at 100% precision by upto 8.99 percentage points over the baseline, while improving reconstruction quality on mixed-scene inputs. Regarding the multi-session mapping task, Graph3R improves cross-session localization and achieves competitive pose estimation accuracy, while delivering a runtime speedup of approximately $35.9\times$ over the baseline when reconstructing the query group. By integrating learned view association with geometric inference, our approach takes a step toward end-to-end reconstruction from uncurated image collections.