Context aware image generation improves reasoning and structure
Context-Aware Autoregressive Image Generation for Emerging Reasoning Properties
Computer Vision and Pattern Recognition
Summary
Generating images one piece at a time often follows fixed or random patterns, which can miss important parts of the picture. The authors propose a new method where the computer chooses what part to draw next by looking at the whole image and what it has already created. This approach helps the model better understand the structure and meaning behind the image, making its creations more accurate and closer to instructions. The research shows that paying attention to where to generate next improves both the image quality and the ability to follow complex directions.
What this means in practice
- •For visual content creators: Generate detailed images that better follow complex semantic instructions by adapting generation order to image structure and context.
- •For design software developers: Integrate context-aware generation to improve the structural consistency of AI-generated images in tools for creative design.
Authors
Jixuan Ying, Haoyu Liu, Timing Yang, Tingyu Zhu, Predrag Neskovic, Anand Bhattad, Cihang Xie, Zeyu Zheng, Alan Yuille, Feng Wang
Abstract
Autoregressive image generation has recently emerged as a competitive visual generation paradigm, yet its generation order is often predefined or heuristic, such as next-patch prediction, random-order prediction, or one-step generation for specific scales. We observe that these strategies can cause models to overlook important structural dependencies in images, thereby limiting both structural robustness and reasoning capability. In this work, we propose a context-aware autoregressive generation framework, where the model dynamically determines the next token location based on global context and current local information. Rather than following a fixed or random order, our method allows the generation process to adapt to image structure and semantic dependencies. Extensive experiments show that context-aware generation significantly improves structural robustness across different image AR paradigms, including masked autoregressive and scale-wise autoregressive models. Beyond quantitative improvements, we further observe emerging reasoning properties: the model can better understand complex semantic instructions and generate images that more faithfully follow the given guidance. These findings suggest that generation order is a critical yet underexplored factor in image autoregressive modeling, and that context-aware decoding can unlock stronger reasoning-aware visual generation capabilities. Code is available at https://github.com/Hpesojyjx/Saccades-AR.