Summary
Controlling how AI generates images often requires understanding hidden features inside the model, but knowing what features represent doesn’t guarantee they can be used to influence the image output. The authors introduce D-Scope, a system that connects the interpretation of these features to actually steering image generation by matching visual patterns to text descriptions. This approach helps test how activating or suppressing certain features changes the generated image and provides a way to inspect and control AI image creation more precisely. They tested D-Scope on many features and conditions to understand how well the features can be used to guide generation.
What this means in practice
- •For image generation engineers: Test and optimize which internal model features can steer AI image generation for better control of visual output.
- •For creative ai developers: Develop more interpretable and controllable AI tools that let users manipulate parts of generated images by linking features to textual concepts.
Authors
Xinyue Xu, Jiahao Zhang, Lijie Hu, Peter Hase, Hao Wang
Abstract
Sparse autoencoders (SAEs) reveal visual structure in diffusion transformers (DiTs), but interpreting a feature does not establish whether it can be used to control generation. We introduce D-Scope (Diffusion Scope), a framework that connects feature interpretation to generation control through shared visual evidence. D-Scope aggregates SigLIP~2 embeddings of highly activating image patches into visual centroids. Matching target text descriptions against these visual centroids in the shared image-text embedding space then enables retrieval of individual features without per-feature text annotations. The underlying patches provide evidence for inspecting each selection, while spatially masked interventions test the corresponding decoder direction at varying strengths under fixed generation conditions. We characterize 150 SAEs across two model families and five layers, and introduce a benchmark of 100 target concepts with ten contexts each spanning under-specified and explicit-conflict conditions. Our empirical results show that high reconstruction fidelity can coexist with low dictionary utilization and limited visual-evidence coverage. Under per-case best-of-sweep strength selection, contrastive retrieval yields larger mean regional SigLIP~2 gains than direct retrieval across the tested steering configurations, without consistently improving outside-region preservation. D-Scope provides an inspectable framework for evaluating sparse DiT features through their visual evidence and the effects of their decoder directions on generation. The demo is available at https://jiahaozhang-public.github.io/d-scope/.