LLM agents often pick items from favored sources even if worse quality
Source Preference in the Wild: How LLM Agents Favor Items by Source, and How to Reduce It
Computation and Language
Summary
When AI helpers pick things like products, hotels, or papers for users, they tend to prefer items from some websites or sources over others. The authors studied many AI models and found that this source preference often outweighs how well an item fits the user's needs. For example, an item that meets fewer requirements will often be chosen if it comes from a favored source. The authors also showed that hiding or changing source labels can reduce this bias, and giving the AI more information or prompts can help it make fairer choices.
What this means in practice
- •For e-commerce platform engineers: Improve product recommendation fairness by reducing source preference in AI-based search and selection algorithms.
- •For online travel agency developers: Adjust hotel booking agents to avoid bias toward certain providers, ensuring choices better match user requirements.
Authors
Jonghyun Song, Haewon Park, Jeonghoon Shim, Woojung Song, Yohan Jo
Abstract
As LLM agents decide on users' behalf which product to buy, which hotel to book, or which paper to cite, a preference for items from certain sources (the sites or services they come from) shapes what users receive and which sources are selected. We study source preference in end-to-end search with 12 agent models across three domains. Comparing items from different sources that satisfy the same requirements at the same position, we find that each model prefers some sources and avoids others in every domain, largely agreeing on which. This preference can outweigh how well items satisfy the request: an item satisfying one requirement fewer is selected about two-thirds of the time when it comes from a preferred source and the better one from a dispreferred source, but almost never in the reverse case. The information identifying an item's source affects selection by itself: hiding it weakens the preference, and relabeling an item with a preferred source raises its selection rate. We test two routes to this preference: training that rewards better items can make a source a shortcut for requirement satisfaction, and missing information can trigger preconceptions about the source. Supplying missing information or a prompt countering these preconceptions reduces source preference.