Fashion image captions improved by retrieval augmented attribute reasoning
RA-CoA: Training-free Fashion Image Captioning via Retrieval-Augmented Chain-of-Attributes
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Giving detailed descriptions of fashion images is hard because it needs attention to small details like patterns and styles. The authors show a way to create accurate fashion captions without needing to train new models. Their method looks up relevant fashion attributes from a product database first, then uses those to guide the caption writing. This approach works with existing vision-language models and makes captioning much more precise without extra training.
What this means in practice
- •For e-commerce product teams: Generate precise and detailed product descriptions for rapidly changing fashion catalogs without custom model training.
- •For fashion retail marketers: Create tailored and accurate product captions that reflect fine fashion details to improve personalized recommendations and search.
Authors
Abhirama Subramanyam Penamakuri, Shreya Shukla, Anand Mishra
Abstract
Fashion Image Captioning (FIC) plays a vital role in enhancing user experience and product search in e-commerce platforms. Unlike natural scene image captioning, FIC requires fine-grained visual reasoning and knowledge of domain-specific terminology to capture subtle attributes such as neckline and closure types, graphic patterns, and dress silhouettes. Moreover, as fashion inventories evolve rapidly with new trends, styles, and frequently emerging vocabulary, developing training-free captioning solution becomes essential for scalability and real-world adaptability. Instruction-tuned vision-language models (VLMs) offer a promising solution to fashion image captioning dueto their strong zero-shot capabilities and natural language fluency. However, these general-purpose models often lack attribute-level coverage and precision, and tend to hallucinate or misidentify fine-grained fashion details, making them less suitable for high-fidelity applications like product cataloging or personalized recommendations. To address this, we propose RA-CoA (Retrieval-Augmented Chain-of-Attributes), a novel, training-free framework that disentangles fashion image captioning into two interpretable stages: (i) retrieval of relevant attribute sets from a product knowledge base, and (ii) attribute-level reasoning to generate the final caption. RA-CoA is a model-agnostic approach that works with frozen VLMs to improve fine-grained attribute precision in product captions without the need for fine-tuning. Extensive evaluations across diverse VLM model families under different prompting paradigms demonstrate that RA-CoA significantly improves caption quality, achieving an average gain of 26.3% METEOR score over zero-shot captioning. We make our code publicly available.