Super-resolution network delivers multiple scales faster using transformer
Progressive Multi-Scale Super-Resolution via Cascaded Bicubic Anchors and Inter-Scale Mutual Refinement
Computer Vision and Pattern RecognitionMachine Learning
Summary
Super-resolution means making blurry images clearer, but most networks can only improve images by one zoom level at a time, needing separate models for different zooms. The authors created a single network, called PMS-SR, that produces clearer images at 2x, 4x, and 8x zoom in one run, using a Transformer model working on the low-resolution input. Their method builds each level progressively from the last using improved math tricks and lets the different zoom levels help refine each other’s results. Tested on standard benchmarks, PMS-SR performs better or as well as other methods while using fewer resources and running faster.
What this means in practice
- •For mobile app developers: Integrate the PMS-SR model to provide multiple image zoom levels efficiently in photo editing or camera apps.
- •For video streaming engineers: Use the model to upscale low-resolution video frames on the fly at several scales with reduced latency and computational cost.
Authors
Chun-An Lin, Yu-Chia Chang, Chi-Mao Fan, Yen-Chieh Ouyang, Tsung-Jung Liu
Abstract
Super-resolution networks are trained for one magnification factor, so several output resolutions require several models. We propose PMS-SR, a progressive single-input multiple-output network that produces x2, x4, and x8 results in one forward pass, with a Transformer backbone on the low-resolution grid. In Stage 1, each scale is predicted as a residual over the detached bicubic upsampling of the previous scale's prediction. With Dirac and zero initialization, training starts exactly from the bicubic cascade, and each head adds only the band the previous scale lacks. In Stage 2, inter-scale mutual refinement couples the scales in both directions through space-to-depth feedback, deformable alignment, and per-pixel fusion, so x8 features refine the x2 output. Trained on DF2K, the 7.95M-parameter model achieves a higher PSNR than every compared multi-scale method at x4 and x8 on all five standard benchmarks. At x8, it outperforms the x8-specific DRN-L on all of them, leads all compared methods on Urban100, and is within 0.13 dB of the best result elsewhere. Against three single-scale SwinIR-M models, PMS-SR uses 22% of the parameters, runs 2.9x faster on BSD100 inputs, and its outputs agree 6.7-12.4 dB more closely. Code and pretrained models will be released at https://github.com/julian135707/PMS-SR upon acceptance.