Adaptive steering improves safe and controlled image generation
Steering Fields: Adaptive Vector Fields for Safe Image Generation and Beyond
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Generating images from text can sometimes create unwanted or harmful content, and controlling this process can be tricky. The authors introduce Steering Fields, a new way to guide image generation that changes its direction step-by-step to keep content safe without ruining the picture. This method works better than previous ones because it adapts to what’s happening during creation and keeps details intact, somewhat like smart image editing. It also doesn’t need extra information about objects or scenes to work well.
What this means in practice
- •For ai product developers: Implement safer text-to-image generation pipelines that prevent harmful outputs while preserving image quality through adaptive steering techniques.
- •For digital content creators: Use Steering Fields as a flexible, model-agnostic tool for precise, structure-preserving semantic image editing without complex setup or priors.
Authors
Simone Facchiano, Jan Eric Lenssen, Bernt Schiele, Wolfgang Stammer, Fabio Galasso, Jonas Fischer
Abstract
As state-of-the-art text-to-image flow models achieve near-photorealistic quality, controlling their outputs, e.g., suppressing harmful content while promoting benign alternatives, has become a central challenge. The current steering paradigm consists of adding a global steering vector to selected activations. While functional, a fixed and example-agnostic vector applied uniformly along the entire trajectory cannot adapt to the changing state of the generation and often causes unintended global changes. We introduce Steering Fields, a generalization of steering vectors that adaptively re-estimates the steering direction at each step of the generative process. Steering Fields operate on the noisy states of flow models, expose a continuous trade-off between steering strength and content preservation, and are compositional, enabling the simultaneous induction and inhibition of concepts, setting a new state of the art on safety steering benchmarks. Despite using no explicit spatial masks or object priors, the trajectory-adaptive estimation naturally preserves local structure, in a manner reminiscent of image editing. In fact, Steering Fields can serve as a structure-preserving image-editing technique that achieves state-of-the-art semantic fidelity (CLIP, VQAScore), while remaining model-agnostic and inversion-free.