PosePlaner improves 3D pose estimates from noisy initial predictions
PosePlaner: Denoising Any Feed-Forward Pose Predictor from Pairwise Planar Geometry
Computer Vision and Pattern Recognition
Summary
Estimating the position and angle of objects in 3D space from images is tricky, especially when predictions are made quickly in one step. The authors created PosePlaner, a method that refines these initial guesses by checking them against simple geometric clues, like matching points between two images. This refinement improves accuracy and fixes mistakes without needing to retrain the system. It works well both for pairs of images and long sequences, helping reduce cumulative errors over time.
What this means in practice
- •For robotics engineers: Enhance robot navigation systems by improving pose estimates with geometric refinement in streaming multi-view inputs without retraining.
- •For augmented reality developers: Improve the accuracy of 3D object tracking in AR applications by reducing drift and correcting pose errors using PosePlaner as a refiner.
Authors
Yihang Chen, Bin Tan, Yujun Shen, Yinghao Xu, Nan Xue
Abstract
Recent multi-view Transformers (e.g., DUSt3R and VGGT) have advanced 3D reconstruction by scaling training data and model capacity from two views to many views and long sequences, subsuming the classical SfM/SLAM pipeline of feature extraction, matching, and pose estimation in a single feed-forward pass. Yet this subsumption hides the interfaces between pipeline stages---correspondences, geometry, and pose are entangled in a single forward pass, leaving supervised fine-tuning as the major path to improvement and offering no mechanism to correct predictions from geometric evidence at test time. We propose PosePlaner to close this gap by reviving the classical coupling between correspondences and pose via flow matching, treating pose refinement as a conditional generation problem that takes coarse geometric matches and initial pose as input condition and fuses the resulting pose hypotheses into a single refined estimate, providing a continuous, data-driven analog of RANSAC voting. Our method is trained efficiently on two-view data, yet serves as a periodic refiner for streaming n-view pose estimators with no retraining. Empirically, it corrects failures of feed-forward predictors, substantially improves correspondence-based solvers, and reduces accumulated drift in long streaming sequences.