AI summaryⓘ
The authors present ARM, a model that uses a special way to turn images into sequences of tokens, allowing it to understand, create, and edit images by predicting the next token. They first train a tokenizer to capture important image features that align with language and reconstruct images well. Then, they train a large autoregressive model on combined text and image tokens to handle vision and language tasks together. Finally, they improve the model using reinforcement learning, making it better at generating images from text and editing images as instructed, with improvements showing benefits across tasks. This work suggests that using token prediction with good representations and preference training can effectively handle multiple visual and language tasks.
autoregressive modeldiscrete visual tokenizernext-token predictionvision-language modelreinforcement learningtext-to-image generationimage editingsemantic representationpreference optimization
Authors
Junke Wang, Xiao Wang, Jiacheng Pan, Xuefeng Hu, Feng Li, Jingxiang Sun, Chaorui Deng, Zilong Chen, Yunpeng Chen, Kaibin Tian, Matthew Gwilliam, Hao Chen, Danhui Guan, Kun Xu, Weilin Huang, Zuxuan Wu, Haoqi Fan, Yu-Gang Jiang, Zhenheng Yang
Abstract
This paper introduces ARM, a discrete representation-based AutoRegressive Model that unifies image understanding, generation, and editing within a next-token prediction framework. ARM is built on three efforts: first, we train a discrete semantic visual tokenizer that maps images into compact token sequences. Our tokenizer is supervised with multiple objectives that jointly promote semantic discriminability, language alignment and faithful reconstruction, thereby supporting diverse tasks in a shared latent space. With this, we train a 7B autoregressive model over large-scale text and image token sequences, seamlessly developing vision-language perception and generation capabilities. Finally, to further improve preference-aligned behavior for text-to-image generation and instruction-guided editing, ARM applies reinforcement learning (RL) to optimize task-level objectives such as visual quality, instruction adherence, and edit consistency. Surprisingly, the results show that RL not only substantially improves performance on the target tasks (e.g., raising WISE overall from 0.50 to 0.56, GEdit-Bench-EN G_O from 5.75 to 6.68), but also induces cross-task synergy between text-to-image generation and editing. Collectively, these findings highlight autoregressive modeling, when paired with strong representations and preference optimization, as a scalable foundation for multimodal intelligence. Code: https://github.com/wdrink/ARM.