Latent diffusion allows detailed local control of image style and content

Local Content-Style Control for Diffusion-based Image Stylization

GraphicsComputer Vision and Pattern Recognition

Summary

Image stylization often changes the whole picture at once, making it hard to fix or improve just one part. The authors found a way to control how each part of an image looks and what it shows during style changes, all in one step. This system uses existing parts but lets users tweak settings for different areas, like changing the style in one area while keeping the content in another. This approach doesn’t need to retrain the model and works with current tools right away.

What this means in practice

  • For digital artists: Edit specific regions of an image with precise style or content changes without affecting the whole picture in one generation pass.
  • For video game designers: Create region-specific texture stylizations that keep content consistent while varying artistic style in game assets.

Authors

Amir Semmo

Abstract

Image stylization with latent-diffusion models entangles two independently refined axes: what a region depicts and how it is depicted. Such pipelines expose only global controls, yet professional retouching demands deliberate, region-specific control. We lift two conditioning weights already present in a ControlNet + IP-Adapter stylization pipeline from global scalars to per-location spatial maps, yielding local, per-axis control of content and style in a single generative pass. Because the two weights act on disjoint pathways, adjusting them independently spans a 2x2 retouching vocabulary, from free regeneration to identity preservation. We validate that edits stay confined to the retouched region and that each weight predominantly steers its own axis. Our approach requires no retraining and drops unchanged into any such pipeline.