Papers for

product recommendation teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Simple method improves picking best items from noisy choices

A Simple and General Approach to top-$k$ Selection

Abstract: This paper considers the Bayesian top-$k$ identification problem, wherein a decision maker is tasked with selecting $k$ items from a pool of items of unknown quality. To assess the items, the decision maker adaptively allocates a fixed sampling budget, where each sample of an item yields a noisy observation of its quality, with the objective of minimizing the expected opportunity cost of the selected set. We propose an approach that allocates each batch of samples by minimizing a surrogate objective derived from an upper bound on the expected opportunity cost. The approach is general: samples can be allocated in batches, the selection size $k$ is arbitrary, and it applies to many sampling distributions. It is also simple and intuitive. We prove that the method's posterior expected opportunity cost decays at an exponential rate of at least half the best rate achievable by any sampling rule. In simulations across a wide range of settings, the method is competitive with, and often improves upon, the best existing algorithm in each setting.

Mon 19 OctMachine Learning
The gist
Sometimes you need to pick the best few items from a group when you can only check their quality a limited number of times, and each check has some noise or error. The authors introduce an easy-to-understand method that decides how to split these checks among items to find the top choices with fewer mistakes. Their method works for any number of selected items and different types of quality measurements. They also prove their method is close to the best possible and show through tests that it often does better than existing approaches.
Open → 2610.26407v1

Learning accurate rankings from noisy human comparison feedback

Learning a Ranking from Human Feedback in Log-Concave Random Utility Models

Abstract: We study the problem of recovering the ranking of a fixed set of items according to their unknown numerical utilities. At each interaction with the environment, a learner presents the item set to a human and receives comparative feedback of two types. Under full-ranking feedback, each interaction reveals a noisy ranking of all items, whereas under winner-only feedback, it reveals only the item ranked first. In both settings, we model human feedback using a random utility model with log-concave noise and study the number of observations needed to recover an $ε$-accurate ranking with high probability. This novel criterion tolerates ordering errors only between items whose utilities differ by less than $ε$. For both feedback types, we establish worst-case sample-complexity lower bounds and develop algorithms that match these bounds up to logarithmic factors. Neither algorithm requires knowledge of the noise distribution, while only requiring an upper bound on its variance. Our results show that the ranking problem under winner-only feedback is intrinsically harder by exposing the sample complexity dependence on the minimum winning probability across the item set.

Tue 6 OctMachine Learning
The gist
This paper looks at how to figure out the order of a set of items based on how humans compare them, even when those comparisons have some randomness or noise. Sometimes, the human gives a full ranking of all items; other times, they just pick the top item. The researchers show how many examples are needed to get a ranking that’s close enough to the true order, and they create methods that work well even without knowing the exact noise details. They also find that learning from only the winner is harder than learning from full rankings.
Open → 2610.07973v1

Large language models show varied brand recommendations with marketplace cues

Evaluating Brand Retrieval and Ranking in Large Language Model Recommendations

Abstract: Large language models (LLMs) are increasingly used for product recommendation, but evaluating their recommendations presents challenges that differ from conventional information retrieval and recommender systems. LLMs can generate recommendations without an explicit candidate set, and repeated responses to the same query can produce different brands and rankings. We introduce a framework for evaluating open-ended LLM brand recommendations that defines the competitive set independently of model outputs and estimates recommendation prevalence and prominence through repeated sampling. We operationalize these constructs using Brand Recommendation Probability (BRP@$k$) and Mean Reciprocal Rank (MRR@$k$), and apply the framework to six LLMs across five product categories. Category-only queries reveal substantial omission of established brands and limited evidence that recommendation prominence follows conventional brand popularity. Instead, prominence is associated with broader marketplace-visibility signals, particularly search interest and online brand conversation. Needs-based queries show that contextualizing users' goals and constraints changes which brands are retrieved, while diagnostic positioning probes demonstrate that brands omitted from ordinary recommendations can remain conditionally retrievable when distinctive cues are supplied. These findings highlight the need to evaluate LLM recommendation as a stochastic retrieval-and-ranking process rather than from individual generated lists. We provide open-source software and data to support reproducible evaluation of LLM-generated brand recommendations.

Mon 14 SeptInformation Retrieval
The gist
Large language models can suggest different brands and rank them differently each time they are asked, making it hard to evaluate their recommendations. The authors created a way to measure how often and how prominently brands appear in these suggestions by repeatedly sampling the model’s outputs. They found that the brands recommended are not always the most popular ones, but often those with high online visibility. The researchers also showed that asking more detailed questions changes which brands are suggested, and some rarely recommended brands can appear when specific hints are given.
Open → 2609.16304v1