CORE: A Unified Cascaded Ordinal Relevance Estimation Framework for E-commerce Search
2026-07-27 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors address a problem in online shopping searches where the system ranks items by relevance. Instead of treating relevance as unrelated categories, they propose breaking it down into a series of yes/no questions about how relevant an item is, going from the most to less relevant levels. They apply this method to improve both large language models and faster BERT models used online. Tests show their approach makes the search rankings better and reduces errors by nearly 16%. Their findings also highlight that handling relevance step-by-step helps improve predictions.
ranking relevanceordinal classificationmulti-class classificationbinary classificationlarge language modelsBERTsequential decision processdistillationA/B testinge-commerce search
Authors
Zhi Jin, Xi Wang, Yunfei Li, Guojun Liu, Qingsong Hua, Wei Lin
Abstract
Ranking relevance is a fundamental task in e-commerce search, directly affecting ranking quality and consumer experience. Although inherently an ordinal classification problem, it is commonly formulated as conventional multi-class classification, which overlooks the natural order among relevance levels and assigns equal penalties to adjacent and distant misclassifications. This mismatch leads to suboptimal learning objectives for practical relevance evaluation. To address this issue, we propose a unified cascaded binary classification framework applicable to both large language model inference and online BERT-based inference, which reformulates relevance estimation as a sequential decision process and decomposes multi-class prediction into a series of ordered binary judgments from higher to lower relevance tiers. For large language models, we design a step-wise reasoning procedure with pruning strategies and tier-specific reward functions. For the online BERT model, we replace the conventional classification head with multiple level-wise binary classifiers and distill the capabilities of large language models into the online model. Extensive offline industrial benchmark evaluations and online A/B experiments demonstrate that the proposed framework substantially improves relevance performance, reducing the online bad-case rate by 15.94\%. Further analyses suggest that tier-wise modeling is effective for relevance estimation.