VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion
2026-07-29 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionRobotics
AI summaryⓘ
The authors made a new system to figure out camera settings and positions from any video, even when the camera is moving wildly or the video is complicated. They combine ideas from two common methods: SLAM, which is fast but can fail easily, and SfM, which is more flexible but can struggle with tricky visuals. Their system uses clues from how the video frames are ordered and adds extra depth information to improve accuracy. Tests show their method works better on tough videos than previous methods, whether the camera details are known or not.
Camera calibrationMetric posesSimultaneous Localization and Mapping (SLAM)Structure-from-Motion (SfM)Loop closureMonocular depth estimationGlobal optimizationWide-baseline matchingVisual symmetriesVideo reconstruction
Authors
Zador Pataki, Paul-Edouard Sarlin, Marc Pollefeys
Abstract
Accurately recovering the camera's calibration and metric poses for any unconstrained video would unlock large-scale training data for navigation and scene understanding. The dominant approaches to this problem are severely limited: Simultaneous Localization and Mapping (SLAM) is sensitive to initialization and transient failures due to its causal, incremental nature; it is often over-optimized for real-time operation and generally requires known camera calibration; while Structure-from-Motion (SfM) typically forgoes any image ordering, enabling optimal initialization and global optimization, but lacks robustness to visual symmetries and extreme motions. To bridge this gap, we introduce a system that combines the strong sequential constraints of SLAM with the flexibility and global optimization of offline SfM, enabling the metric reconstruction of arbitrary, long, uncalibrated videos. This system leverages recent advances in wide-baseline dense image matching, treats temporal ordering as a first-class citizen for reliable loop closure, and augments global optimization with metric monocular depth priors. As a result, thorough evaluations on diverse, challenging datasets that exhibit extreme motion and visual symmetries reveal that our approach is significantly more robust and accurate than both state-of-the-art SLAM and SfM, classical or learned, with given or unknown camera calibration. The code is publicly available at https://github.com/cvg/vidmap.