Geometric inconsistency detection improves wide baseline image analysis
Geometric Inconsistency Localization in Multi-View Image Sets
Computer Vision and Pattern RecognitionArtificial IntelligenceCryptography and SecurityMultimedia
Summary
When computers create new pictures of a scene from different viewpoints, sometimes the images don’t line up properly, causing geometric mistakes. The authors made a new collection of image pairs with marked geometric errors to study these problems. They found that current methods, which work well for generating new views, don’t do well at finding where exactly the errors are in images taken from very different angles. To fix this, they built a special tool that learns to spot these mismatches better, reducing mistakes and pinpointing problem areas in the images.
What this means in practice
- •For forensic analysts: Detect and pinpoint geometric manipulation in images captured from different viewpoints to improve multimedia authenticity verification.
- •For augmented reality developers: Use accurate geometric inconsistency detection to enhance overlay reliability when combining images taken from several wide baseline angles.
Authors
Xander Staelens, Albéric Loos, Bert Ramlot, Hannes Mareen, Peter Lambert, Glenn Van Wallendael
Abstract
Novel view synthesis (NVS) models can produce realistic new views of the same scene from different viewpoints. However, these generated views are not always geometrically consistent with one another. Multi-view (MV) consistency has shown promise as a tool for evaluating these NVS models. Its potential for multimedia forensics, however, remains largely unexplored, particularly for localizing geometric inconsistencies across wide-baseline image pairs. To enable research in this direction, we introduce DeformView, a wide-baseline MV dataset with pixel-level annotations of geometric inconsistencies. Using DeformView, we evaluate state-of-the-art MV consistency-scoring methods and show that approaches developed for NVS evaluation transfer poorly to the forensic task of geometric inconsistency localization. To address this limitation, we propose DEFECt3R, a lightweight learning-based classifier that uses cross-view feature relationships to localize geometric inconsistencies at the pixel level. By learning from explicit supervision, including hard negatives from geometrically consistent yet deformed views, DEFECt3R improves localization performance and substantially reduces false positives compared to existing consistency-scoring methods. Ablation experiments further show that both feature representations and correspondence quality contribute to localization performance. Overall, our findings demonstrate that MV geometric consistency is a promising yet underexplored signal for multimedia forensics and establish a benchmark and baseline for geometric inconsistency localization in wide-baseline MV image pairs. Code and dataset are available at https://github.com/IDLabMedia/DeformView-DEFECt3R