Language models help decide where to place product recommendations on search pages

Language Models for Page-Level Layout Decisions in E-commerce Search

Machine LearningComputation and LanguageInformation Retrieval

Summary

Online shopping sites often show extra product recommendations alongside the main search results. Putting these recommendations in the right place helps shoppers find what they want and keeps them interested, but putting them in the wrong place can make things confusing. The authors studied how to use language models—computer programs trained to understand text—to predict if placing extra recommendations at certain spots will be helpful, without testing on real shoppers. They found that using language model features as numeric representations works better for this than just using direct prompts. This method could help design better shopping pages without expensive online tests.

What this means in practice

Authors

Varun Joshi, Eva C. Song, ChengXiang Zhai

Abstract

E-commerce search pages are critical touchpoints for millions of online shoppers. While traditional search engines return a ranked list of results, modern E-commerce search pages increasingly incorporate recommender system modules -- for example, secondary stacks that surface alternative product groupings at specific positions. When introduced appropriately, secondary stacks can improve user engagement; however, suboptimal placement may disrupt browsing flow and degrade the primary results. Unlike traditional search ranking, where evaluation techniques such as interleaving are well established, evaluating page-level layout changes e.g., when and where to insert a secondary stack remains challenging without costly online A/B testing. To address this, we study offline methods for evaluating whether a given layout decision -- specifically, the inclusion of a secondary stack at a particular position -- is beneficial to users. We investigate language models as scalable evaluators by comparing direct prompt-based, prompt-derived feature, and representation-based methods. Our results show that representation-based approaches consistently outperform prompt-based judging in predicting user engagement, suggesting they provide a reliable foundation for offline layout evaluation in E-commerce search.
View PDFOpen arXiv