ScaleLUT accelerator boosts real-time video upscaling on edge devices
ScaleLUT: A Fully-Parallel Configurable LUT-Based Accelerator for Real-Time Multi-Scale Super-Resolution
Hardware Architecture
Summary
Making low-resolution videos look better and sharper on devices like phones or TVs in real time is hard because the usual methods need lots of math and power. The authors developed ScaleLUT, a new system that uses clever lookup tables and hardware design to do this upscaling quickly and with less energy. This system works for different zoom levels, supports very high resolutions like 4K, and runs much faster and more efficiently than previous methods on real hardware. This approach combines smart algorithm design with hardware to make improved video clearer and more practical for everyday gadgets.
What this means in practice
- •For mobile device engineers: Build energy-efficient video enhancement features that upscale videos to higher resolutions in real time on handheld devices without draining battery.
- •For broadcast hardware designers: Integrate a configurable accelerator that performs fast 4K video super-resolution at various zoom levels for live TV and streaming equipment.
Authors
Boyu Li, Chenchen Ding, Zhilin Ai, Wenqing Shi, Baizhou Jiang, Wenyong Zhou, Binxiao Huang, Jiachen Ren, Hao Yu, Ngai Wong
Abstract
Real-time super-resolution (SR) remains challenging for edge devices because deep-learning-based methods require substantial multiply-accumulate (MAC) operations, resources, and power. Lookup-table (LUT)-based SR reduces computation by replacing convolutional inference with table queries, but existing methods still suffer from limited speed, large storage overhead, and poor scalability across upsampling factors. We present ScaleLUT, a hardware-oriented LUT design framework and fully parallel reconfigurable accelerator for real-time multi-scale SR. ScaleLUT combines a hardware-friendly YUV-domain strategy with power-of-two kernels and rotation ensemble to improve receptive-field coverage while reducing LUT dimensionality; division operations are replaced by shifts. These designs reduce memory by 18.4% over state-of-the-art LUT-based SR methods. ScaleLUT supports arbitrary input resolutions and configurable x2^n upsampling factors using a deeply pipelined, massively parallel architecture. Implemented on a Xilinx ZCU102 FPGA, it achieves real-time 4K SR at 95.3 FPS for x2 upscaling at 300 MHz. Compared with existing SR accelerators, ScaleLUT uses at least 58.6% fewer LUTs, 41.1% fewer flip-flops, zero DSPs, and 42.0% lower power, while delivering 10x and 1.2x speedups over the best CPU-based SR implementation and prior FPGA-based SR accelerators, respectively. These results demonstrate the effectiveness of joint LUT algorithm-hardware co-design for practical and energy-efficient edge SR deployment.