User simulators improve task focus and sound more natural in conversations
Towards User Simulators That Stay On Task And Still Sound Natural
Computation and Language
Summary
AI assistants often lose track of what a user wants during long chats, making the conversation less helpful. The authors created a new user simulator that better sticks to the user's goals while sounding more like a real person. They used a special training method that balances staying on task and sounding natural by testing the simulator against another model that judges naturalness. This new simulator, called UserLM-2.0, produces conversations that are both on task and natural much more often than older models.
What this means in practice
- •For chatbot developers: Build more realistic and goal-focused simulated users to better test and train AI assistants for long conversations.
- •For customer support platform teams: Improve training environments to handle complex user requests consistently while keeping interactions natural and engaging.
Authors
Mustafa Omer Gul, Philippe Laban, Tanya Goyal, Jennifer Neville
Abstract
Users increasingly rely on AI systems for extended, multi-turn tasks, yet evidence indicates that LLM-based system performance degrades over prolonged interaction. To evaluate and improve AI assistants in such settings, we need user simulators that faithfully follow complex user intents while exhibiting realistic conversational behavior. Existing approaches manifest a trade-off: prompted assistants can follow complex user intents but lack behavioral fidelity, while specialized user simulators better approximate conversational behavior but struggle to remain faithful to complex intents. In this paper, we propose a reinforcement learning (RL) recipe that jointly rewards task intent adherence and behavioral naturalness. A crucial contribution of the method is operationalizing naturalness with a lightweight discriminator trained in an alternating, adversarial loop with the user simulator. We release the resulting model, UserLM-2.0, which produces valid simulated conversations (i.e., meeting both task-faithfulness and naturalness criteria) 46.2% of the time, over three times the rate of the strongest prior user simulator (14.1%).