Efficient image compression runs on limited FPGA hardware at high speed
VQ-LIC: Shared Vector-Quantized Learned Image Compression on a Resource-Constrained FPGA
Hardware Architecture
Summary
Compressing images using AI usually needs powerful hardware, but the authors created a method that works well on small, limited devices called FPGAs. They built a system where the simple parts run on the small device while more complex tasks happen in the cloud. Their design reuses hardware efficiently and precisely predicts how fast it will run. This results in fast, low-energy image compression with good quality, even on modest hardware.
What this means in practice
- •For embedded system engineers: Build real-time image compression modules on resource-limited FPGA hardware for edge devices with predictable performance and low power.
- •For cloud infrastructure teams: Support edge-cloud image codecs where lightweight encoding runs on edge FPGAs and detailed decoding occurs in the cloud for bandwidth savings.
Authors
Muhammad Fahd Ibrahim Bhatti, Abdullah Bin Faisal, Ahsan Usman, Naveed Anwar Bhatti, Muhammad Ali Siddiqi
Abstract
Learned image compression (LIC) is hard to deploy on severely resource-constrained FPGAs, since how fast it actually runs depends not just on arithmetic count, but also on memory traffic, imbalance between different operations, and how the hardware batches its work. We present VQ-LIC, an asymmetric edge-cloud codec in which a compact INT8 depthwise (DW)-pointwise (PW) analysis transform and multi-codebook vector quantization (VQ) run at the edge on a reusable DW/PW engine pair, while reconstruction is handled by a larger cloud decoder. Since VQ codeword matching is expressible as a dot product, it is mapped directly onto the same PW engine, removing the need for a separate VQ compute array, to our knowledge a first for FPGA LIC. A novel latency model, derived from deterministic RTL cycle counts of an FPGA's read, DW, PW, and write costs, predicts an analysis transform's per-block latency; since VQ shares the same PW datapath, the model applies to VQ as well. Validated directly against silicon, the model predicts deployed analysis and VQ latency within 0.26\% and 0.05\%, and guides the selection of a three-block $16$-$48$-$64$ transform. Post-training codebook reduction then cuts VQ arithmetic and codebook storage by $4\times$ and shrinks the fixed-width latent representation. On a 220-DSP Zynq-7020, VQ-LIC's mid-rate preset reaches 0.1398 bits per pixel at 28.69 dB PSNR and 13.06 dB MS-SSIM on CLIC~2017, outperforming a similarly sized neural encoder and reaching a rate-distortion range comparable to a codec three orders of magnitude larger. The complete 0.1945-kMAC/pixel analysis-VQ pipeline runs at 47.98 frames per second and 42.84 mJ per frame on silicon, using an order of magnitude fewer DSPs than comparable FPGA LIC accelerators while achieving lower bitrate, higher throughput, and lower energy per frame at a modest PSNR tradeoff.