GeoCond improves reliability awareness in fast 3D scene reconstruction
GeoCond: A Conditioning-Aware Reliability Adapter for Feed-Forward 3D Reconstruction
Computer Vision and Pattern Recognition
Summary
Feed-forward 3D models quickly create 3D maps and camera views but sometimes quietly make big mistakes when the camera angles or overlaps are tricky. The authors show these errors depend on certain geometric conditions and that the models' own confidence scores don't catch these problems well. They developed GeoCond, a small add-on that examines the 3D model's output and estimates how reliable it is, helping decide when to trust or improve the results. This method worked better on tests, especially in unusual outdoor situations, and can be used without extra data in many cases.
What this means in practice
- •For augmented reality developers: Improve the reliability estimation for real-time 3D scene reconstructions under challenging camera movements to avoid silent failures during AR experiences.
- •For autonomous vehicle engineers: Use a lightweight adapter to assess and refine 3D environment perception from forward-looking sensors, improving pose accuracy in difficult viewing conditions.
Authors
David Ahmedt-Aristizabal, Mohammad Ali Armin, Russell Tsuchida, Lars Petersson
Abstract
Feed-forward 3D foundation models such as VGGT predict cameras, depth, and point maps in a single pass, but can fail silently under low overlap, low parallax, and extreme relative rotation. Stratified analyses over these factors show that these failures are governed by geometric conditioning and are poorly captured by native aleatoric confidence. We introduce GeoCond, a lightweight reliability adapter for frozen feed-forward 3D backbones. GeoCond reads the backbone's predicted geometry and outputs pose-level uncertainty and a refinement gate. During training, it can be supervised by frame-permutation orbit variance, ground-truth pose error when labels are available, or cycle residuals from unlabelled independent pose graphs. At inference, the default head requires only one backbone pass and a small MLP. On VGGT, GeoCond improves out-of-distribution (OOD) AUSE (area under the sparsification-error curve; lower is better) from $0.32$ to $0.20$ over native confidence, transfers zero-shot to outdoor extreme-view scenes, and avoids the collapse caused by applying bundle adjustment uniformly. Across multiple backbones, cycle-distilled variants provide a ground-truth-free adaptation route, including cases where permutation variance vanishes on equivariant models. The same reliability signal supports gated refinement, pose-graph weighting, calibration, curation, and capture decisions. Reliable feed-forward 3D reconstruction requires not only predicting geometry, but also knowing when that geometry should be trusted.