Graphene improves image super-resolution with efficient feature refinement

Graphene: A Lightweight Transformer with Ising Mean-Field Refinement for Image Super-Resolution

Computer Vision and Pattern Recognition

Summary

Image super-resolution is the problem of making low-quality images clearer and sharper. The authors propose Graphene, a lightweight system that refines image features by using ideas from physics and attention techniques to improve detail reconstruction. It uses a special block inspired by the Ising model to refine local features, combined with a history-based attention that blends current and past features. Graphene achieves better performance than previous methods with less computational cost and memory use, while producing clearer images in real-world tests.

What this means in practice

  • For mobile app developers: Improve photo enhancement features by integrating Graphene for faster, higher-quality image super-resolution on mobile devices with limited resources.
  • For video streaming platforms: Enhance video upscaling algorithms by incorporating Graphene’s lightweight refinement to provide clearer content with reduced computational demand.

Authors

Seungki Hong, Taekwang Jang

Abstract

This paper proposes Graphene, a lightweight image super-resolution architecture with two complementary feature refinement mechanisms: Ising-model-inspired local feature refinement using an Ising Refinement Block (IRB) and stage-history-based feature refinement using a History Attention (HistAttn) module. The IRB reuses attention output features and pre-softmax attention logits to construct signed pairwise token couplings within each local window, applies unrolled mean-field updates, and refines features through a gated residual correction. These signed couplings allow alignment and opposition between latent spins to modulate local feature refinement. HistAttn attends over the current stage output and earlier representations at the same token position and adaptively blends the current stage output with a history-weighted feature aggregate without adding spatial pairwise attention. On Urban100 at $\times4$, Graphene and Graphene-light improve over IET and IET-light by 0.24 dB and 0.38 dB, respectively, with 34.3%/13.7% fewer parameters, 44.8%/44.4% fewer FLOPs, 78.4%/71.5% lower latency, and 76.0%/85.5% lower peak memory. Real-world SR comparisons further show that Graphene recovers fine details and preserves textures across diverse degradations. Code and reference checkpoints are available at https://github.com/skethz/Graphene