Step distilled models enable style transfer via text instructions
Salvation Lies Within: Eliciting Inherent Style Transfer in Step-Distilled Diffusion Models
Computer Vision and Pattern Recognition
Summary
Making computers create stylish images usually means training them on lots of examples, which can be expensive and slow. The authors found a way to get already-trained image generators, called step-distilled diffusion models, to mimic different artistic styles just by giving smart text instructions. They developed a method called StyleForge that automatically turns example images into detailed style descriptions. This lets the model redraw many new images in that style without extra training and still keep the speed and quality it was built for.
What this means in practice
- •For digital artists: Create artwork in a specific style on demand without retraining the model, enabling fast and flexible artistic rendering from new prompts.
- •For graphic design teams: Produce styled images across different content themes using the same underlying model and style description, improving workflow efficiency.
Authors
Shengyin Sun, Yiming Li, Yingzhao Lian, Xing Li, Xingzhi Zhou, Anxin Tian, Zhili Wang, Haoyang Li, Ziqiang Cui, Chen Ma
Abstract
Adapting step-distilled text-to-image (T2I) models through post-training incurs additional computational costs and affects native few-step generation behavior. This motivates a complementary route beyond style-specific adaptation: drawing on the visual knowledge already encoded in step-distilled T2I models to elicit stylistic capabilities through language. Pursuing this direction requires textual guidance that captures how visual attributes jointly define a style and remain applicable as the depicted content changes. To explore this approach, we introduce StyleForge, a fully automatic, training-free framework that expresses reference styles as reusable rendering instructions. By integrating overall rendering characteristics with local color and lighting behavior, StyleForge organizes visual evidence from reference images into a coherent specification of how the target style should be expressed. The specification is then compiled into textual guidance that can be reused across content prompts, enabling frozen step-distilled T2I models to render different subjects and scenes in the reference style while retaining native few-step generation. Extensive experiments show relative gains of up to 29.47\% in generation quality scores over the strongest baseline, while Pareto analysis indicates that improved stylization is accompanied by strong adherence to the requested content.