UniGen-AR: Unifying Visual Generation with Auto-Regressive Modeling
2026-07-27 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem that current computer vision models are specialized and separate for tasks like image generation, editing, and restoration. They propose UniGen-AR, a single model combining a multi-modal language model with an efficient visual auto-regressive decoder to handle many image-related tasks using one system. This approach is much faster than popular diffusion-based models while keeping the quality high. Their experiments show benefits from careful design choices in the underlying image tokenizer, which help the model scale well. Overall, they demonstrate that auto-regressive models can be an efficient way to unify different visual generation tasks.
Unified Visual GenerationDiffusion ModelsAuto-Regressive ModelsMulti-Modal Language ModelsVQ-VAECodebookInference LatencyImage TokenizerImage GenerationVisual Transformer
Authors
Zhipeng Bao, Zhen Zhu, Nupur Kumari, Anurag Bagchi, Yu-Xiong Wang, Pavel Tokmakov, Martial Hebert
Abstract
Modern computer vision pipelines remain fragmented, with tasks such as text-to-image generation, editing, restoration, and classical perception handled by separate models. We study Unified Visual Generation (UVG), where a single model produces diverse image-valued outputs through a unified multimodal interface. While diffusion-based systems dominate UVG due to strong quality and controllability, their iterative sampling incurs substantial inference latency, limiting practical deployment. To address these limitations, we propose UniGen-AR, a framework that pairs a general-purpose multi-modal language model (MLLM) with an efficient next-scale visual auto-regressive (VAR) decoder. This design retains the flexibility of MLLM-based conditioning while leveraging the sampling efficiency and latent unification properties of VAR models. In our framework, the MLLM encodes free-form instructions and control signals into a unified sequence, which guides the VAR decoder to generate image-valued outputs for over 15 tasks spanning four families. Empirically, UniGen-AR achieves up to $19 \times$ lower inference latency than diffusion-based baselines while maintaining or improving output quality. Our ablations further reveal that VQ-VAE tokenizer design, particularly codebook size and hierarchy, is a critical factor for VAR scalability in UVG. These results establish visual auto-regressive modeling as a compelling and efficient backbone for unified visual generation. Our project page is at https://zpbao.github.io/projects/unigenar.