Evaluating models for improving real-time endomicroscopy image alignment
Evaluating Transformation Models for pCLE Mosaic Registration
Computer Vision and Pattern Recognition
Summary
Confocal Laser Endomicroscopy (CLE) lets doctors look at cells inside the body in real time, but each image covers only a small area. To see a bigger picture, the images need to be stitched together exactly, which is tricky because the tissue can move and change shape. The authors created a dataset with marked points to measure how accurate different image alignment methods are. They found that simple techniques don’t handle tissue movement well, and that more flexible models work better, though some advanced AI methods showed only slight improvements. Also, aligning pairs of images well does not always guarantee good quality for the entire stitched image.
What this means in practice
- •For medical imaging developers: Improve image stitching algorithms for endomicroscopy devices by choosing transformation models and matchers that better handle tissue deformation.
- •For surgical navigation teams: Enhance real-time cellular-level imaging by integrating validated registration methods to provide larger view mosaics during procedures.
Authors
Ahmed Aboelela, Johannes Barcsay, Jana Friedhof, Miguel Gonçalves, Alexander Hann, Katharina Breininger
Abstract
Confocal Laser Endomicroscopy (CLE) provides real-time, cellular-resolution optical biopsy but has a narrow field of view, which image mosaicing can extend to provide anatomical context. Because of line-by-line acquisition, probe motion, and probe-tissue interaction, frame alignment generally requires a non-linear transformation whose accuracy is difficult to quantify: flexible transformation models can fit intensity features and noise, so appearance-based metrics such as Normalized Cross-Correlation (NCC) can improve without a genuine gain in geometric accuracy. We therefore establish a dataset of 132 frame pairs across fourteen pCLE sequences from 4 patients with manually annotated landmark correspondences, so that Target Registration Error (TRE) can serve as a geometrically grounded complement to NCC. We assess the effect of progressively increasing the transformation model's degrees of freedom, from translation to Thin Plate Spline (TPS), and of six feature-matching backends spanning classical (Shi-Tomasi, Lucas-Kanade) and learned (SuperPoint, SuperGlue, LightGlue, LoFTR, RoMa) approaches. Translation and rigid models prove insufficient under tissue deformation, while TPS with random sampling achieves the strongest landmark-derived alignment of the evaluated configurations; among the learned matchers, used without fine-tuning, only RoMa offers a robust, if modest, advantage over other methods. At the sequence level, pairwise registration quality proved an unreliable predictor of final mosaic quality, so mosaic quality must be evaluated directly rather than inferred from pairwise metrics.