Efficient multimodal models match large AI for search relevance prediction
From a Large-Scale Teacher to Efficient Multimodal Students for Search Relevance Prediction
Information RetrievalMachine Learning
Summary
Predicting how well a search item matches a user’s query is important but usually needs expensive human reviews. The paper’s authors created a system where a big AI model teaches smaller, faster models to do this task. They first made sure the big model is reliable by checking it with quality human data. Then, they used the big model to label lots of examples, training the smaller models to perform almost as well but much faster and cheaper. This method makes it easier to use good search prediction in real online stores.
What this means in practice
- •For e-commerce platform engineers: Use compact multimodal models for fast, cost-effective search relevance scoring without losing accuracy compared to large language models.
- •For mobile app developers: Integrate efficient relevance prediction models to improve in-app product searches within limited compute constraints.
Authors
Tomasz Gniazdowski, Mohammadreza Momenifar, Nadia Vase, Shahram Khadivi, Wiktor Walentynowicz, Nicola Ueffing, Jan Pawlowski, Piotr Szymanski, Abraham Bagherjeiran
Abstract
Relevance prediction between user queries and items is a fundamental component of e-commerce search systems. Traditionally, high-quality relevance labels are obtained through human judgment, which is reliable but expensive, slow, and difficult to scale. Recent advances in large language models (LLMs) enable automated relevance judgment at scale, achieving performance comparable to human annotators. However, such systems often rely on large commercial LLMs with complex prompts, resulting in high inference costs and limited throughput. In this work, we propose a framework that bridges this gap by distilling a high-performing LLM-based relevance judgment system into efficient multimodal student models. We first curate a small, high-quality human-annotated dataset to serve as ground truth. Using this dataset, we develop a reliable LLM-as-a-judge pipeline through careful prompt engineering and iterative refinement. This pipeline is then used to generate large-scale relevance labels, which serve as training data for compact multimodal models. Our distilled models achieve performance close to the teacher system while significantly improving latency, cost, and scalability. This approach enables practical deployment of high-quality relevance prediction systems in production environments.