New method improves recognition of unseen image compositions
From Model Patterns to Abstract Semantics in Compositional Zero-Shot Learning
Computer Vision and Pattern Recognition
Summary
Recognizing new combinations of objects and their features in images is hard because models usually learn fixed examples. This paper introduces a method called CLEAR that treats parts of images as clues activated by context, similar to how humans perceive things. CLEAR uses a step-by-step reasoning process to better understand and rank possible interpretations of images, reducing errors from obvious visuals. Tests show CLEAR consistently enhances existing models and beats other methods on tough image datasets.
What this means in practice
- •For computer vision engineers: Improve image recognition systems by enabling them to identify novel object combinations they haven't seen before.
- •For machine learning system builders: Enhance AI models in robotics or autonomous systems to better interpret complex visual scenes with unfamiliar object states.
Authors
Weize Li, Zhicheng Zhao, Fei Su
Abstract
Compositional Zero Shot Learning aims to recognize unseen compositions by recombining learned primitives. Recent methods rely on vision language models and attempt to explicitly model contextual variations of primitives through multiple representations. However, such approaches are limited by fixed variant capacity and competition between abstract and concrete semantics. In this work, we present a new perspective that views primitive variations as the context-driven activation of concrete visual cues rather than independent entities. Based on it, we propose CLEAR, a CLoze-style rEAsoning-based Re-ranking framework inspired by human perceptual processes. CLEAR extracts conditional variants from the primitive candidate set in a coarse-to-fine manner, performs cloze-style reasoning to infer high-level semantics, and re-ranks predictions to correct biases toward salient concrete primitives. Extensive experiments demonstrate that CLEAR consistently improves the Base Model and outperforms state-of-the-art methods on the challenging C-GQA and MIT-States datasets. Code is available at https://github.com/buptLwz/CLEAR.