Summary
When you zoom in or out on a smartphone using two cameras, the pictures can suddenly change in ways that look strange or jarring. The authors define a task called dual-camera smooth zoom (DCSZ) to create smooth, continuous zoom transitions by generating frames between the two camera views. They found that existing methods don't work well because of big differences between camera views and lack of real training data. So, they made a new way to simulate training videos by separating how the scene's shape and colors change. They also built a special network that mixes information from both cameras to create smoother zoom videos and showed it works better than existing methods.
What this means in practice
- •For mobile app developers: Create smoother zoom effects in smartphone camera apps by generating intermediate frames between dual-camera zoom views.
- •For camera hardware engineers: Improve the visual consistency of zoom transitions in dual-camera smartphone hardware by integrating better interpolation models trained on synthetic data.
Authors
Junjie Fan, Zhilu Zhang, Renlong Wu, Dazhi Zhang, Wangmeng Zuo
Abstract
When zooming between dual cameras on smartphones, prominent discontinuities in both geometric perspective and color rendition inevitably degrade the visual experience of users. To address this issue, we introduce the dual-camera smooth zoom (DCSZ) task, aiming to synthesize continuous intermediate frames for fluid zooming. However, directly applying existing video frame interpolation (VFI) models to this task is severely hindered by substantial domain gaps and the practical intractability of capturing real-world continuous ground-truth. To this end, we propose a novel data synthesis framework to generate high-fidelity DCSZ training sequences, named ZoomGS v2. It explicitly decouples geometric and color transformations from 3D scene representation modeling, thereby enabling extensive data generation from easily accessible single-camera 3D scenes. Furthermore, we develop ZoomFI, a specialized VFI network tailored to the DCSZ task, which leverages complementary bidirectional warping for comprehensive dual-camera information fusion. Extensive experiments on test datasets captured using two mobile phones demonstrate that training with the constructed DCSZ data significantly improves the performance of various VFI models. Moreover, ZoomFI achieves state-of-the-art results in both quantitative metrics and visual quality. The datasets, code, and pretrained models are available at https://hitjunjief.github.io/DCSZ-v2