LoopVAE reduces parameters for image tokenization with recurrent depth
LoopVAE: Recurrent Depth Across Scales for Visual Tokenization
Computer Vision and Pattern Recognition
Summary
Encoding images efficiently can require a lot of computer memory and calculations. The authors introduce LoopVAE, a system that reuses parts of its processing steps multiple times across different image sizes to cut down on the number of unique parts needed. This reuse leads to fewer parameters while keeping image quality good on a standard dataset. The study also compares different designs and shows that repeating processing steps can have meaningful effects on the final image reconstruction.
What this means in practice
- •For computer vision engineers: Build more memory-efficient image encoding models by reusing core computations across scales, reducing model size for deployment on constrained devices.
- •For machine learning system architects: Design image analysis pipelines that balance runtime and parameter storage by implementing recurrent sharing of processing blocks in hierarchical visual models.
Authors
Zhiying Lu
Abstract
Hierarchical visual tokenizers typically allocate different processing blocks to different spatial scales. We ask how much of this computation can use the same parameters. LoopVAE reuses a scale- and loop-conditioned core within and across scales, while keeping resolution-changing transitions independent. A four-block core executes 28 block applications per encoder or decoder. On ImageNet-256, the 29M-parameter convolutional model reaches 0.28 rFID and 32.54 dB PSNR under an approximately 30-epoch two-stage training budget, using approximately 65% fewer parameters than the 84M reference VAEs. A non-adversarial Transformer ablation with the same execution graph finds competitive PSNR and SSIM under global sharing, although unshared blocks improve LPIPS. Targeted loop interventions show that completing the trained recurrence improves reconstruction and that even small feature updates can have substantial downstream effects. Truncation also exposes output-range errors, distinguishing useful recurrent computation from reliable early exit. Runtime profiling reveals the execution tradeoff: fewer stored weights require more arithmetic and longer runtime in the tested configurations. With convolutional and Transformer operators and single- or multi-resolution latent interfaces, LoopVAE establishes recurrent depth across scales as a parameter-sharing design axis for visual tokenization.