DORS: Dynamic Attention Routing for Diffusion-based Object Removal in Dense Scenes
Computer Vision and Pattern Recognition
Summary
The authors address the problem of removing objects from busy scenes where similar-looking items nearby confuse existing methods, causing incomplete removal. They propose a new system called DORS that uses special attention techniques to filter out confusing information and better use context for a cleaner result. They also created a new test set, DOR-Bench, to measure how well methods work in these tricky scenes. Their experiments show DORS works better than current methods, especially in avoiding leftover objects and duplicate artifacts.
Authors
Haitong Tang, Haipeng Liu, Yang Wang
Abstract
Object removal aims to eliminate target objects specified by a mask while preserving visual consistency with the surrounding regions. Existing methods typically rely on contextual information from surrounding regions. However, in dense scenes where the surrounding regions contain instances visually similar to the removal target, such reliance often leads to semantic interference, resulting in incomplete removal. This problem arises from erroneous information propagation in the attention space, where masked queries tend to align with such instances due to global similarity matching in self-attention. To address this challenge, we propose a Diffusion-based Object Removal framework for dense Scenes, dubbed DORS, built upon a Dynamic Attention Routing mechanism comprising two complementary components: Instance-Filtered Attention (IFA), which suppresses misleading semantic information from similar instances through dynamically constructed mask-guided attention constraints, and Context-Guided Routing (CGR), which dynamically routes complementary scene information to maintain visual consistency. We further introduce DOR-Bench, a benchmark tailored for object removal in dense scenes. Extensive experiments demonstrate that DORS outperforms state-of-the-art methods, particularly in reducing incomplete removal and duplicate artifacts. The code will be available at https://github.com/httang1224/DORS.