Flexible feature matching improves image correspondence and rejection
FlexGlue: Sparse-to-Dense Feature Matching and Visual Disambiguation
Computer Vision and Pattern Recognition
Summary
Matching points between two images is key for tasks like 3D mapping, but current methods either match sparse points quickly or match every pixel slowly. The authors introduce FlexGlue, a method that finds sparse points in one image and searches a few promising spots densely in the other, combining speed with accuracy. It also learns to say when two images don’t actually match, reducing errors caused by confusing patterns. Tests show it outperforms earlier methods in estimating camera positions and matching scenes.
What this means in practice
- •For augmented reality developers: Improve real-time camera tracking accuracy by matching image points quickly and correctly rejecting irrelevant scenes using FlexGlue.
- •For robotics navigation teams: Use FlexGlue to enhance 3D mapping by accurately identifying corresponding points between images while avoiding false matches from symmetrical visuals.
Authors
Philipp Lindenberger, Paul-Edouard Sarlin, Ondrej Miksik, Marc Pollefeys
Abstract
To estimate correspondences across images, existing approaches match either sparse interest points, which is highly efficient but limited by repeatability, or dense pixel grids, which is robust but expensive. Furthermore, these approaches are trained with only matching pairs and thus often fail to reject unrelated pairs that exhibit visual symmetries, which is equally important in applications like Structure-from-Motion (SfM). We bridge these gaps by introducing FlexGlue, a neural network for asymmetric sparse-to-dense matching and disambiguation. Given two images and coordinates of points in the first image, FlexGlue estimates the coordinates of the corresponding points in the second image while simultaneously rejecting the pair if non-matching. By searching for a limited set of correspondences among dense pixels, our approach is both fast and effective at both matching and disambiguation: it combines the sub-pixel accuracy of dense matching with the speed of sparse matching, while rejecting incorrect pairs with high precision. Experiments on multiple datasets show that FlexGlue pushes the Pareto frontier in relative pose estimation and significantly outperforms existing approaches for SfM and visual localization. FlexGlue is publicly available at https://github.com/cvg/flexglue .