Object removal improved in few-step diffusion models using state correction
ReState: Noise-Level-Aware State Correction for Training-Free Object Removal in Few-Step Diffusion Models
Computer Vision and Pattern Recognition
Summary
Removing objects from images using fast diffusion models can leave behind faint traces of the removed object. The authors found this happens because the model predicts the masked area early on and has limited steps to fix it later. They introduce ReState, a method that adjusts the image prediction inside the masked region using noise-level information from the original image. This approach improves image quality without extra training or slower processing, working well across different diffusion model versions.
What this means in practice
- •For photo editing software developers: Integrate training-free object removal that works faster and produces cleaner results in photo editing tools using few-step diffusion backbones.$Commercial implications: Enables commercial photo editing products to offer faster and higher-quality object removal features without extra model training.
- •For mobile app developers: Implement efficient object removal in mobile apps that use distilled diffusion models, improving speed and visual quality under computational constraints.
Authors
Donghwan Lee, Sanghyun Jo, Sihyun Kim, Ziseok Lee, Kyungsu Kim
Abstract
Training-free object removal commonly redirects self-attention away from the target region, but on few-step distilled diffusion models this often leaves object-shaped residuals. We identify a temporal mismatch: a structured prediction forms inside the mask at the first forward, and the short trajectory offers few later chances to revise it, while attention rerouting acts mainly in later steps. Across four distilled backbones, magnitude-matched single-forward interventions show that background-directed state correction affects the final result more than attention rerouting. We therefore propose ReState, which corrects the masked clean prediction toward background evidence retrieved from the input using a noise-level-aware weight that vanishes at the final forward. ReState retains object-excluding attention as a complementary constraint and adds no denoiser evaluations. Across two paired benchmarks and two few-step backbones, ReState achieves the best masked PSNR, SSIM, and LPIPS among training-free methods, improving PSNR by 3.17 dB on average over the strongest baseline. On OBER-Test, it also outperforms the 50-step attention-forcing reference on all three metrics at 6.7x lower latency. The same configuration transfers to two additional distilled backbones without retuning. Project page: https://tdrlee.github.io/ReState