AcCoRD: Evaluating User-Agent Collaboration Under Realistic User Preference Dynamics
Artificial Intelligence
Summary
The authors explain that when people work with AI assistants, their likes and dislikes often change during the conversation, but most tests for these assistants only look at fixed preferences. They created AcCoRD, a new benchmark that tests AI on handling changing user preferences in shopping and travel planning. They tried five top AI models with two ways of asking questions, and found the models can deal with uncertainty at first but struggle when preferences change later on. The authors highlight that just prompting the models isn't enough for spotting changing preferences and offer AcCoRD to help improve smarter AI assistants.
Authors
Tejas Srinivasan, Shikib Mehri, Nandita Shankar Naik, Anirban Das, William M. Campbell, Jesse Thomason
Abstract
User preferences in user-agent collaboration are rarely static and fully-specified upfront: preferences are formed, revealed, adjusted, and relaxed during interaction. Existing benchmarks for evaluating user-agent collaboration focus almost exclusively on resolving underspecified preferences, thereby failing to capture the richer dynamics of real-world interaction. We introduce AcCoRD, a user-agent collaboration benchmark requiring agents to handle diverse user preference dynamics in two domains: online shopping and travel planning. We evaluate five frontier LLMs under two prompting strategies: vanilla ReAct, and an uncertainty-guided variant that prompts models to identify and resolve ambiguity about user preferences. Our results reveal that frontier models can handle underspecification but struggle to satisfy preferences that emerge or evolve mid-interaction and require more sophisticated uncertainty modeling. Further, prompting alone fails to elicit the required uncertainty recognition. We release AcCoRD as a resource for developing agents that can navigate the full complexity of real-world user preferences.