AI agents show bias based on who assigns their role in shopping advice
Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders
Artificial Intelligence
Summary
AI shopping assistants need to advise buyers honestly but are often hired by platforms that earn money from ads. The authors found that when these AI agents are told to work for a seller instead of the buyer, they are less skeptical of paid product placements. This means the AI may favor promoted listings more when acting on behalf of the platform. The usual warnings that tell buyers which items are sponsored don’t fully protect shoppers when AI is involved.
What this means in practice
- •For e-commerce platform designers: Design AI shopping assistants that account for role-induced bias to ensure fair recommendations regardless of sponsorship.
- •For advertising compliance teams: Improve sponsorship disclosure practices for AI agents to prevent hidden bias that benefits the platform over consumers.
Authors
Davood Wadi, Yu Ma
Abstract
Large language models (LLMs) now serve as conversational shopping assistants on platforms that also sell advertising. These AI agents face a conflict of duty. They advise consumers who rely on their judgment, yet are deployed by platforms that benefit when sponsored listings are chosen. Sponsorship disclosures, designed to allow consumers to penalize paid placements, now reach the AI agent rather than the consumer, and the agent's evaluation of them is hidden from the consumer. Drawing on the fiduciary concept of conflict of duty, we argue that an agent's evaluation of a sponsored listing should not depend on which party deployed it. In controlled choice experiments, we manipulate assigned roles in the system prompt to name either a traveler or a booking platform as the agent's principal. Platform delegation significantly attenuates the penalty that agents apply to sponsored listings and weakens the skepticism that disclosure triggers in their reasoning traces. We replicate out findings across LLMs and reasoning depths. A second study decomposes the disclosure label and shows that the divergence between the two delegates widens significantly when the paid placement is attributed to the platform. Stricter terminology ("Sponsored" instead of "Promoted") lowers choice of paid listings but does not close this gap when the platform is named. The findings show that disclosure mandates designed for human consumers cannot by themselves protect consumers in AI-mediated commerce.