Diffusion model fine-tuning makes multi-target domain adaptation efficient
Revisiting Diffusion Fine-Tuning for Unsupervised Domain Adaptation
Computer Vision and Pattern Recognition
Summary
Sometimes, computer programs need to learn from one set of labeled examples and then work well on many different new types of data without labels. Previously, adapting these programs for each new type took a lot of extra work and time. The paper studies a new approach that fine-tunes the learning process once and then customizes parts of it for each new type, saving time and computer resources. This method produces better results across different new data types while being more efficient.
What this means in practice
- •For machine learning engineers: Train models that adapt efficiently to multiple new unlabeled data types by reusing a single fine-tuned diffusion process.
- •For data platform teams: Reduce compute and storage costs for generating target-specific synthetic data for multiple domains from a single source dataset.
Authors
Xuan Qi, Yi Wei, Daniele Berardini, Vito Paolo Pastore, Vittorio Murino
Abstract
Diffusion-based unsupervised domain adaptation (UDA) improves cross-domain transfer by generating target-specific synthetic data for downstream adaptation. Existing methods are largely designed for single-target adaptation: when a model trained on one labeled source domain must be adapted to multiple unlabeled target domains, they typically require separate diffusion fine-tuning for each source--target pair, causing training, storage, and deployment costs to grow with the number of targets. In this paper, we study multi-target data generation for diffusion-based UDA, where a single source-guided diffusion fine-tuning process is reused to generate target-specific synthetic data for multiple target domains. We propose MUSE (Multi-target UDA-oriented Synthesis with Efficient diffusion fine-tuning), a decoupled adaptation framework that separates source-supervised semantic adaptation from target-specific style adaptation. MUSE uses a shared semantic branch updated by labeled source data and target-private style branches specialized to individual target domains, enabling target-specific generation while avoiding repeated source-guided fine-tuning for each target. Experiments on standard UDA benchmarks show that MUSE achieves a stronger accuracy--efficiency trade-off than repeated per-target diffusion adaptation, reducing diffusion fine-tuning cost while improving average target-domain accuracy. The project page is available at https://xuanqi99.github.io/MUSE/.