Conversational shopping agents improved with real ecommerce data
RealWorldShop: Benchmarking and Improving Conversational Shopping Agents in Real-World E-commerce
Artificial Intelligence
Summary
Online shopping is becoming more interactive, with language models helping shoppers by holding conversations to find the right products. Current systems often fail to understand changing user needs and keep track of the shopping session properly. The authors created a huge new testing setup called REALWORLDSHOP with millions of real products and simulated shopping conversations. They also built a new agent, REALSHOP_AGENT, which manages the shopping session better and gives more relevant recommendations throughout the chat.
What this means in practice
- •For ecommerce platform developers: Create better conversational shopping assistants that handle full sessions with evolving user preferences and multiple goals.$Commercial implications: Enables selling interactive shopping agents that improve customer engagement by understanding complex intent and providing product-grounded recommendations.
- •For customer service chatbot builders: Improve chatbot systems to track customer constraints and update recommendations dynamically during shopping conversations.
Authors
Xinwei Yang, Kelong Mao, Yudong Guo, Sulong Xu, Simiu Gu, Chen Huang, Wenqiang Lei
Abstract
Large language models are reshaping ecommerce from static recommenders into interactive shopping assistants, yet real-world shopping requires session-level decision support: users reveal and revise constraints, coordinate multiple goals, and expect product-grounded recommendations over a full conversation. Existing benchmarks are mostly outcome-oriented or execution-oriented, leaving this evolving decision process under-evaluated. We introduce REALWORLDSHOP, a benchmark built on 3.28M grounded products, structured shopping episodes, a profile-grounded and actioncontrolled user simulator, and role-play evaluation. Our analysis shows that current systems produce locally plausible responses but struggle with state tracking, constraint updating, and grounded convergence, especially under ambiguous intent, bundle, and multi-intent scenarios. We further propose REALSHOP_AGENT, an executable session-control framework with explicit state management, shopping-flow control, catalog-grounded retrieval, and runtime guards. Experiments show that REALSHOP_AGENT consistently outperforms strong baselines on REALWORLDSHOP.