RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration
2026-06-29 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the hard problem of matching images from different sensors, which can look very different and are hard to align accurately. They propose RBE-Flow, a new method that treats this matching like a repeated guessing and correcting game, using a smart way to estimate both the best alignment and how sure it is about that guess. Their method uses feedback on uncertainty to improve its corrections over time, helping it avoid mistakes in tricky areas. Tests on multiple datasets show their approach works better than previous methods, especially when very precise alignment is needed.
cross-modal image registrationBayesian estimationfeature manifoldsnon-linear optimizationuncertainty estimationrecurrent neural networkssigma-point projectionprobabilistic state updatedense flow estimationsub-pixel accuracy
Authors
Mengzhu Ding, Xin Song, Xiaoke Ding, Hongwei Ding, Xuecong Liu
Abstract
Cross-modal image registration is essential for multi-sensor perception but remains fundamentally challenging due to severe non-linear radiometric discrepancies and geometric distortions. Existing deterministic matching methods lack uncertainty awareness, struggling to navigate the resulting highly non-convex optimization landscape and frequently accumulating errors in ambiguous regions. In this paper, we propose RBE-Flow, a novel framework that reformulates dense cross-modal flow estimation as a closed-loop recurrent Bayesian estimation problem on learned feature manifolds. Diverging from standard feed-forward regression, RBE-Flow establishes a robust self-correcting mechanism by deeply coupling feature-metric non-linear optimization with probabilistic state updates. Specifically, a Recurrent Manifold Optimization (RMO) block iteratively generates flow observations and their associated uncertainties, which are then optimally assimilated into the prior state via an Uncertainty-Adaptive Probabilistic Update (UAPU) using deterministic sigma-point projection. Crucially, the resulting calibrated posterior covariance is fed back to adaptively regularize the damping of subsequent optimization steps, allowing the system to modulate its convergence based on predictive confidence. To ensure stable probabilistic training, we introduce a hybrid supervision scheme featuring a geometry-aware rectified NLL loss that structurally prevents variance collapse. Extensive experiments on challenging OSdataset, WHU-OPT-SAR, and RoadScene benchmarks demonstrate that RBE-Flow consistently achieves state-of-the-art performance, outperforming existing methods by a significant margin, particularly under strict sub-pixel criteria. Project page: https://github.com/NEU-Liuxuecong/RBE-Flow