Loom: Hybrid Retrieval-Scoring Outfit Recommendation with Semantic Material Compatibility and Occasion-Aware Embedding Priors
2026-05-11 • Information Retrieval
Information RetrievalComputer Vision and Pattern Recognition
AI summaryⓘ
The authors developed Loom, a system that helps pick complete outfits by combining smart search techniques with rules about fashion. Starting from one clothing item, Loom finds matching pieces using a special fashion embedding and scores outfit choices based on things like color, style, and occasion fit. They introduced new ways to guess how heavy clothes are and how well they fit an event without manually coding those details. Tests show their full system works much better than random choices, especially thanks to a step that orders outfits by style. Loom can quickly suggest three different outfits in under 5 seconds.
neural embedding retrievalFashionCLIP embeddingsapproximate nearest neighbor searchmulti-objective scoringcolor harmonyformality consistencyoccasion coherencesemantic material weightstyle directionvibe/anti-vibe occasion priors
Authors
Anushree Berlia
Abstract
We present Loom, an outfit recommendation system that combines neural embedding retrieval with structured domain scoring to generate complete, coherent outfits from fashion catalogs. Given an anchor clothing item, Loom retrieves complementary pieces via slot-constrained approximate nearest neighbor search over FashionCLIP embeddings, then scores candidate outfits using a multi-objective function that integrates six signals: embedding similarity, color harmony, formality consistency, occasion coherence, style direction, and within-outfit diversity. We introduce two techniques that address limitations of purely learned or purely rule-based approaches: (1) semantic material weight, which uses CLIP embedding geometry to infer garment heaviness for layer compatibility without hand-coded material taxonomies; and (2) vibe/anti-vibe occasion priors, which embed prose descriptions of occasion contexts as anchor vectors in CLIP space and score items by differential affinity. Ablation experiments on a catalog of 620 items show that each component contributes measurably to outfit quality: the full system achieves a mean outfit score of 0.179 with a 9.3% hard violation rate, compared to 0.054 score and 16.0% violations for a category-constrained random baseline, a 3.3x improvement in score and 42% reduction in violations. Direction reranking is the single indispensable component: removing it drops score to 0.052, essentially equal to random. The system generates three stylistically distinct outfits in under 5 seconds on commodity hardware.