DiCE-CIR: Direct Composition Learning for Efficient Zero-Shot Composed Image Retrieval
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address zero-shot composed image retrieval, which means finding a new image based on a given image plus a description of how to change it, without needing example triplets during training. Instead of converting images into text tokens, their method, DiCE-CIR, directly combines the image and text to create a search query. They generate training examples automatically using a large language model and train a simple module to understand these combinations. Their experiments show that DiCE-CIR performs very well compared to previous methods while being faster and more efficient.
Zero-shot learningComposed image retrievalMultimodal queryImage captioningLarge language modelRepresentation learningImage-text alignmentSemantic transformationComputational efficiency
Authors
Gwang-Ho Na, Ho-Joong Kim, Seong-Whan Lee
Abstract
Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image from a multimodal query consisting of a reference image and an edit text describing the desired modification. Recent ZS-CIR studies have relied on projection-based methods that map a reference image into pseudo-word tokens in the text embedding space. However, such methods require additional projection and re-encoding steps, increasing training complexity, reducing efficiency, and introducing a discrepancy between training and inference. In this paper, we propose DiCE-CIR, a direct composition learning method that predicts composed query representations by directly composing a reference image and an edit text. To enable scalable training without manually annotated triplets, we automatically construct compositional training samples from large-scale image-caption pairs using a large language model. Based on these samples, we train a lightweight composition module with objectives that promote alignment with the target, edit-consistent semantic transformation, and retrieval discriminability. We conduct extensive experiments on ZS-CIR benchmarks and show that DiCE-CIR achieves state-of-the-art performance on CIRCO and competitive performance on CIRR while maintaining high computational efficiency.