Loop flow transformers improve image generation with less computation

LiFT: Loop Flow Transformers

Machine LearningArtificial Intelligence

Summary

Generating high-quality images with AI often requires a lot of computing power. The authors present a new model called Loop Flow Transformers that repeatedly uses the same core process multiple times during generation. Instead of predicting the final image all at once, their method gradually improves the image over several steps, each trained to move closer to the target. This approach allows the model to generate better images with fewer parameters and less computation.

What this means in practice

Authors

Mohammad Mahdi Derakhshani, Pedro M. P. Curvo, Gertjan J. Burghouts, Jan-Willem van de Meent, Cees G. M. Snoek

Abstract

We introduce Loop Flow Transformers (LiFT), a family of looped generative models that scales computation by repeatedly applying a shared Diffusion Transformer (DiT) core, with only light changes to the standard architecture. Rather than asking every recurrent step for the final prediction, LiFT trains each step with a single regression target: a point on a straight path from the model's initial estimate to the flow-matching target. Because we index these targets by a continuous depth coordinate, a trained model can loop far beyond its training depth with no retraining, early exits, or other modifications. In our experiments, these longer rollouts improve generation, so inference computation can grow without adding parameters. On ImageNet at 256x256, LiFT-L/2 achieves an FID 3.34 points lower than our dense DiT-XL/2 baseline while using approximately 60% fewer parameters, 32% fewer training FLOPs, and 52% fewer inference FLOPs.