Multi view local feature matching improves speed and accuracy
MVLoMa: Multi-View Local Feature Matching with Any Geometry Foundation Model
Computer Vision and Pattern Recognition
Summary
Matching points in images helps computers understand 3D scenes from photos, but usually this works only between pairs of images. The authors developed MVLoMa to match one image to many neighbors at once, using geometric knowledge across all views. This approach speeds up matching and improves accuracy by considering multiple images together rather than separately. They tested it on various datasets and showed it works better and faster than existing methods.
What this means in practice
- •For 3d vision engineers: Improve 3D reconstruction pipelines by matching one reference image to multiple neighbors jointly, reducing computation time and increasing pose accuracy.
- •For robotics developers: Enhance robot visual localization by integrating multi-view geometric context into feature matching for more reliable pose estimation in complex environments.
Authors
Xinyang Lu, Shengzhe Wei, Sheng Tang, Weiqi Zhang, Juntong Fang, Zesong Li
Abstract
Local feature matching is typically formulated for image pairs, although structure-from-motion and visual localization commonly match one reference image against many retrieved or covisible neighbours. Applying a pairwise matcher independently to these centre-neighbour edges duplicates reference-side computation and prevents the matcher from exploiting complementary multi-view geometric evidence from the remaining observations. To address these limitations, we introduce MVLoMa, a sparse star-graph matcher that extends LoMa from image pairs to multiple views. MVLoMa first uses a frozen multi-view geometry model to jointly describe all images in a star, then injects the resulting cross-view geometric context into local descriptors. It then matches the centre image to all neighbours in a single forward pass through per-view self-attention and bidirectional centre-neighbour cross-attention. For supervision, we derive positive correspondences only from depth-valid, mutually visible dense warps and explicitly penalize high-confidence matches that violate epipolar geometry. On 25-view star graphs from HyperSim, ScanNet++, Mip-NeRF 360, and Tanks and Temples, MVLoMa obtains the best macro-average relative-pose AUC among the compared methods at all three thresholds (58.65/85.41/90.99). Its matcher is 2.20 times faster than sequential evaluation of the 24 centre-neighbour edges. The same design supports four tested frozen geometry encoders -- DA3, $π^3$, VGGT, and VGGT-$Ω$ -- with DA3-Giant performing best on all reported encoder-ablation metrics. Controlled ablations further isolate the effects of geometry-aware descriptors, visibility filtering, geometric negative supervision, and joint multi-view inference.