Diffusion method improves style transfer while keeping content intact
TRACE: Time-Adaptive Residual Attention Control with Content-Style Decomposition for Training-Free Diffusion Style Transfer
Computer Vision and Pattern Recognition
Summary
Style transfer means changing the look of a picture while keeping its main objects recognizable. Existing diffusion methods that do this often struggle to balance copying style properly without mixing up the details from the style image or losing the original content. The authors introduce a new technique called TRACE that carefully separates the style from content using a special analysis and adjusts the process dynamically during image creation to avoid mix-ups. This makes the style transfer look more faithful to the style while keeping the original image’s structure clearer.
What this means in practice
- •For graphic designers: Create stylized images that retain the original layout without requiring re-training on new styles.
- •For game artists: Generate game textures that adopt new visual themes while preserving key object shapes using a flexible style transfer method.
Authors
Duc Khoan Le, Kim Ngoc Tran, Minh Nhat Le, Thanh An Tran, Viet Toan Nguyen, Khanh An Lay, Tran Thai Son, Hoang Pham Minh
Abstract
Reference-guided style transfer aims to preserve the semantic structure of a content image while transferring the visual appearance of a style reference. Recent diffusion-based methods achieve impressive stylization quality by exploiting strong pretrained generative priors. However, training-free approaches still face a difficult trade-off among style fidelity, content preservation, and content leakage. Direct style injection may unintentionally transfer semantic content from the style image, while fixed guidance schedules often ignore the time- and state-dependent nature of diffusion sampling. To address these limitations, we propose TRACE, a training-free diffusion style transfer framework with Time-adaptive Residual Attention Control and Content-Style Decomposition. TRACE first performs offline CLIP-based subspace analysis to separate content and style directions from paired data. During inference, it removes content-related components from the style reference and style-related components from the content reference to reduce leakage. It then injects style information through residual cross-attention and applies uncertainty-aware guidance to adapt the guidance signal at each denoising step. Experiments show that TRACE achieves a favorable trade-off between stylization and preservation. Compared with optimal-control-based baselines, TRACE substantially improves style fidelity (+17.28 CSD and +34.10 SRA). While, compared with stylization methods, it better preserves content structure (+12.80 DINO, +5.52 CLIP-I, and -8.19 LPIPS) and reduces directional semantic leakage by 29.5% in DCL. Our code is publicly available at https://github.com/pixelchemy-research/TRACE.