Polaris: Scaling Up Instruction-Guided Image Generation Towards Millions of Personalized Style Needs
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created Polaris, a system that helps image generation models quickly meet different user needs without needing to train new models every time. Instead of starting from scratch, Polaris smartly picks and combines from thousands of already trained pieces called models and adapters based on what the user wants. This makes image creation faster, more flexible, and easier to control. Polaris does all this by searching through a huge library of these components and putting together the best fit for the instructions given.
image generation modelsfine-tuningmodel adaptersmodel retrievalmodule integrationinstruction-driven generationscalable AIpersonalized image synthesis
Authors
Zhi-Kai Chen, Jun-Peng Jiang, Jun-Jie Tao, De-Chuan Zhan, Han-Jia Ye
Abstract
Users increasingly expect image generation models to quickly adapt to highly diverse and personalized requirements, such as producing images with distinctive styles or characteristics. Traditional approaches rely on fine-tuning, which is costly and difficult to scale. To cope with these limitations, the community has accumulated a growing library of fine-tuned modules and adapters, where each component targets specific generation needs and collectively serves as a foundation for handling new demands. This naturally raises a question: instead of repeatedly training new models, can we systematically exploit this expanding ecosystem to better fulfill user instructions? To this end, we present Polaris, an intelligent retrieval framework that automatically selects and integrates suitable models from the model library based on a user's instructions. The key insight is that harnessing such a massive and heterogeneous pool requires not only finding the most relevant modules among thousands of candidates, but also aligning them effectively for instruction-driven generation and editing. Polaris addresses this challenge by indexing over 6,500 checkpoints and 75,000 adapters, and retrieving the most relevant components given a user's input and instruction. In doing so, it delivers scalable, controllable, and well-aligned generation -- without any additional training.