Attention guided masking struggles to improve object discovery in images
Does Attention-Guided Masking Really Help Object Discovery in Object-Centric Learning?
Computer Vision and Pattern Recognition
Summary
Object-Centric Learning tries to teach computers to find objects in pictures without being told where they are. Some methods randomly hide parts of the image and ask the computer to guess them, helping it learn better. The authors studied a new way of hiding parts based on what the computer is already paying attention to, hoping it would improve object finding. They found that this attention-based hiding doesn’t always do better than random hiding, especially for finding main objects. It does help a bit with separating background in some realistic images.
What this means in practice
- •For computer vision engineers: Improve background segmentation in realistic image datasets by integrating attention-guided masking with unconditional slot initialization.
- •For machine learning practitioners: Choose random masking over attention-guided masking for better or comparable object discovery during self-supervised object-centric learning tasks.
Authors
Youliang Tao, Yanhua Han, Bin Zhao, Juho Kannala, Joni Pajarinen, Rongzhen Zhao
Abstract
Object-Centric Learning (OCL) aims to decompose images into objects without human annotations. A major family of mainstream methods uses Slot Attention to aggregate image features into object-level representations and then from them reconstructs masked image content, i.e., Random Masking (RM), to provide self-supervision. The recent method DIAS simply masks image patches at uniform randomness yet achieves competitive object discovery accuracy. Since attention during aggregation already possesses object discovery ability, we explore using it to develop a better image patch masking strategy, i.e., Attention Guided Masking (AGM), thereby providing better self-supervision. Results on six recognized datasets show that AGM does not always outperform RM. Under unconditional slot initialization, AGM substantially improves background segmentation on datasets with realistic textures (COCO and VOC); Regardless of conditional or unconditional slot initialization and across datasets, foreground object discovery remains comparable or decreases. We suggest peer researchers in the OCL community that attempts to exploit internal attention semantics to improve OCL with masked decoding are risky. Our source code, model checkpoints and evaluation logs will be released upon acceptance.