Text to image generation gain new control over styles and concepts
AcFlow: Controlling Text-to-Image Diffusion Transformers via Learned Conditional Activation Flow
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Diffusion transformers for turning text into images are very good but hard to control in how strongly a style shows or what unwanted parts appear. The authors created AcFlow, a tool that changes intermediate image data inside the model during its run, guided by a description of what to change. AcFlow can smoothly adjust style intensity and better remove unwanted parts without retraining the model or needing new data for each style. Tests show AcFlow does better than other methods at balancing style and content, and it works even for styles it has never seen before.
What this means in practice
- •For graphic designers: Adjust style intensity of text-to-image generated artwork with fine control and minimal content loss.
- •For advertising content teams: Remove or suppress unwanted visual concepts in AI-generated images where direct text prompts fail.
- •For interactive media developers: Offer users dynamic style controls in applications that generate images from text descriptions.$Commercial implications: Enables commercial software to provide fine-grained, adaptive style controls for user-driven image generation.
Authors
Junran Wang, Zehao Jin, Tianyu Luan, Xinjie Shen
Abstract
Text-to-image diffusion transformers (DiTs) are powerful generators, yet direct prompting provides limited control interface for style intensity and can fail to suppress unwanted concepts. To enable these controls, we introduce AcFlow, an inference-time controller that transports intermediate layer image-token activations through a learned concept-conditioned velocity field while keeping the base DiT frozen. A textual concept description specifies the desired intervention, while the integration horizon provides a continuous control parameter. The field produces token-varying, activation-dependent updates. With parameters shared across concepts within each task family, the field supports fine-grained descriptions and generalizes to concepts unseen during training without per-concept fitting. On style control, AcFlow achieves the best style--content trade-off among the evaluated baselines in the high-style-alignment regime. At a fixed operating point, AcFlow attains style--content alignment of 0.5365/0.2860, compared with 0.4397/0.2684 for the baseline with the highest style alignment. Qualitative results demonstrate suppression of diverse concepts, including cases where direct prompting fails. Our analyses support the learned velocity field as an adaptive control mechanism, with update directions varying across tokens and depend on their activation states. Our code is available at https://github.com/Nove1yst/AcFlow.