Mirage improves 3D tracking accuracy using combined camera and LiDAR data
MIRAGE: Metric Inference and Reliability-Aware Gaussian Estimation from Feed-Forward Vision
Computer Vision and Pattern Recognition
Summary
When creating 3D models from video, systems often struggle to keep correct size scale and to track movement precisely over time. The authors developed MIRAGE, a method that combines camera-based geometry with motion data from LiDAR and inertial sensors to improve accuracy and reduce drift. MIRAGE also adjusts how it corrects movement estimates based on how well data matches across these sensors. The authors tested MIRAGE on several datasets, showing it outperforms established vision-only methods, and used it to create a new benchmark for indoor 3D reconstruction with monocular cameras.
What this means in practice
- •For robotic navigation teams: Improve long-term pose accuracy and metric scale consistency by combining LiDAR-IMU motion with camera-based 3D geometry for reliable indoor mapping.
- •For augmented reality developers: Generate high-fidelity 3D reconstructions with accurate scale using monocular cameras assisted by LiDAR-IMU data to enhance AR scene understanding indoors.
Authors
Xinzhe Wang, Changjian Jiang, Kaiwen Song, Xudong Li, Kerui Ren, Ran Yi, Lizhuang Ma, Chunhua Shen, Linning Xu, Tao Lu, Mulin Yu
Abstract
Feed-forward 3D priors enable high-quality reconstruction from image streams but often lack reliable metric scale and suffer from long-term trajectory drift. In contrast, LiDAR--inertial estimation provides metrically accurate motion but lacks the image-space consistency needed for high-fidelity reconstruction. We propose $MIRAGE$, which anchors feed-forward visual geometry to LiDAR--IMU metric motion and regulates online pose corrections according to correspondence support and cross-modal agreement, jointly enabling accurate metric pose estimation and high-fidelity Gaussian reconstruction. Experiments on Oxford Spires, M2DGR and INS show that MIRAGE achieves accurate metric trajectory estimation, outperforming COLMAP, a widely used but computationally expensive reference for vision-only benchmarks. Motivated by the scarcity of long-horizon monocular benchmarks for large-scale indoor reconstruction, we further build a multimodal data-collection platform and use MIRAGE to establish accurate 6-DoF reference trajectories for a new vision-only benchmark. We benchmark recent monocular large-scale reconstruction methods on this dataset, providing a systematic evaluation of their performance in long-horizon indoor scenes.