Geometric Distillation from Rectified Stereo: Leveraging Epipolar Cues for Monocular Depth
Computer Vision and Pattern Recognition
Summary
The authors found that models estimating depth from a single image often get the size or scale wrong, even if the shape looks correct. Models that use multiple views can better understand scale because they compare images, but this doesn't help when only one image is used later. To fix this, the authors designed a method called Epipolar Distillation that teaches single-image models to keep geometric cues learned from multi-view models. This makes single-image depth predictions more accurate in terms of real-world scale, improving performance on tricky datasets without needing extra images during use.
Authors
Jung-Hee Kim, Xiaoming Liu
Abstract
Monocular depth foundation models have demonstrated remarkable generalization capabilities across diverse environments. However, they continue to struggle with metric depth estimation in diverse environments. This limitation stems from the inherent scale ambiguity of single-view inference, leading to misaligned scale predictions even when the relative geometry is accurate. Conversely, recent multi-view foundation models leverage cross-view cues to learn robust scene-level geometry and consistent scale. Yet, these benefits typically vanish during single-image inference, as the absence of explicit geometric constraints causes performance to degrade. To bridge this gap, we propose a novel framework that transfers the scale-aware geometric priors of multi-view models into monocular depth foundation models. Specifically, we introduce an Epipolar Distillation (EpiDistill), an approach utilizing Rectified Stereo Tokens, which enables the single-view prediction model to retain epipolar attention patterns and maintain geometric consistency without requiring multi-view inputs at inference. Experimental results demonstrate that our method significantly improves zero-shot metric depth estimation, particularly on challenging datasets like ETH3D and DIODE where scale alignment is critical. Furthermore, our approach is model-agnostic, consistently boosting the performance of state-of-the-art ViT-based models, including UniDepthV2 and DepthPro.