Summary
Taking a clear, detailed photo from a wide-angle camera can be difficult because the image might look blurry or low-resolution. This paper presents a way to improve such photos by using a second camera with a telephoto lens that captures more detail from a distance. The authors designed a special method to carefully match and combine information from both cameras, even when the pictures don’t line up perfectly due to differences in perspective or obstructions. Their approach uses training with mixed real and synthetic images to handle tricky cases and intelligently decides how to blend the data for the best final picture. They also created a challenging new dataset to test their method and showed it works better than previous attempts.
What this means in practice
- •For camera system engineers: Enhance wide-angle low-resolution images by reliably combining them with telephoto references despite parallax and occlusion.
- •For mobile phone designers: Improve smartphone dual-lens image quality by integrating a flow estimation and warping method tailored for dual cameras.$Commercial implications: Enables selling smartphones with better photo enhancement features using dual lenses.
Authors
Fangpu Zhang, Huanjing Yue, Zifan Cui, Shaoxiong Tu, Qirui Yang, Jingyu Yang
Abstract
Dual-lens super-resolution (DuSR) aims to enhance a wide-angle low-resolution (LR) image by exploiting a telephoto reference (Ref) image. A central challenge in DuSR is reliable reference utilization under large parallax, occlusion, and cross-camera imaging discrepancies. Existing methods often employ off-the-shelf monocular optical flow models to provide geometric priors. However, their assumptions of photometric consistency and smooth motion are violated in dual-lens scenarios, leading to unreliable alignment and degraded reference-based reconstruction. To address these issues, we propose DuSRFlow, a dual-lens super-resolution framework that improves reference utilization from both correspondence learning and reference aggregation. First, we introduce DuFlowNet, a dedicated dual-lens flow estimator for robust cross-view alignment. To train DuFlowNet without ground-truth flow annotations, we develop a real--synthetic data fusion strategy that composites synthetic foreground objects onto real dual-lens image pairs, generating controllable large-parallax samples while preserving realistic cross-camera imaging characteristics. Second, we observe that reference reliability is highly spatially uneven in DuSR, where unreliable local reference responses may suppress the utilization of reliable cues. We therefore propose a Complementary Reference Warping strategy that coordinates patch-based warping and kernel-free-guided deformable warping according to local reference reliability, producing a more reliable reference representation before LR--Ref fusion. In addition, we construct DuSR-RealV2, a challenging real-world benchmark with complex depth variations, large parallax, and occlusion. Extensive experiments on multiple DuSR benchmarks demonstrate that DuSRFlow achieves state-of-the-art performance in reconstruction quality, perceptual quality, and cross-dataset generalization.