Does Rank Still Matter? Position Bias When AI Agents Shop on Our Behalf
2026-08-24 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors studied how AI agents search through hotel listings compared to humans by randomizing the order of 100 hotels and analyzing searches by four large language models. They found that AI agents look through more listings and always choose to buy, unlike humans. While placement on the list still influences which hotels are checked, middle results are looked at the least, and the importance of position varies between models. Ultimately, all AI models pick the same best hotel, showing that what details are shown matter more than where the listing is placed. This suggests AI searches differently than humans and rely more on attributes than list order.
search rankingsAI agentslarge language modelssequential searchconsumer behaviorrandomizationinspection probabilitychoice stagesearch attributeshotel listings
Authors
Davood Wadi, Yu Ma
Abstract
Search rankings are valuable because human attention is scarce and sequential. Higher-placed alternatives are easier to find, so they are examined and bought more often. Consumers are now delegating search to AI agents that can ingest an entire results page at once. Randomizing the order of one hundred hotel listings across 5,000 AI agent sessions, we compare four large language models against human field data. AI agents search more deeply than humans and never decline to buy. Position still predicts which listings are inspected, but weakly and non-monotonically: the middle of a results page has the lowest probability of inspection, not the bottom. Position reaches the choice stage for some models and not others, a heterogeneity that tracks neither provider nor capability. All models nonetheless converge on the same undominated listing. For agentic search, the attributes displayed on a results page matter more than placement within it.