Proactive agents balance useful work timing and user trust
Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym
Artificial Intelligence
Summary
Sometimes computer systems try to help before you even ask, but doing so can cause problems like misunderstanding what you need or making you lose trust. The authors explain how to build these helpful systems by focusing on three important ideas: doing the right tasks well, deciding when to work based on resources and timing, and keeping users trusting and comfortable. They also created a testing environment called PROACTIVITY-GYM that simulates realistic scenarios to see how well these systems perform. Their tests show it’s important to carefully balance all these parts because even correct help can reduce trust if it interrupts users unexpectedly.
What this means in practice
- •For software developers: develop proactive assistant features that choose when and how to help users while maintaining trust and efficient resource use.
- •For workflow automation teams: assess and improve automated agents that perform tasks in advance to optimize support without disrupting user focus.
Authors
Jio Oh, Seunghyun Do, Young-Jun Lee, Steven Euijong Whang, Dongyeop Kang
Abstract
Proactive LLM agents can turn idle compute into useful support before users ask. Yet even correct work can misread user context, impose review costs, or undermine trust. This work proposes foundations for designing, realizing, and evaluating proactive LLM agents around three joint principles (3T): Task Capability, anticipating relevant needs and correctly performing useful work; Temporal Allocation, allocating compute according to resource availability and when results are needed; and Trust, sustaining users' confidence and appropriate reliance on the agent. We connect these objectives to a design space organized around five dimensions: task scope, anticipation horizon, activation trigger, processing timing, and intervention depth, and specify the situation and system modeling needed to support its choices, including user and environment representations, backbone LLMs, and agent harnesses. Lastly, we propose PROACTIVITY-GYM, a simulation-based evaluation testbed including multi-day scenarios, stateful environments, and persona-conditioned simulated users that can evaluate the consequences of proactive assistance across interactions. Evaluations across 23 model-harness configurations uncover substantial performance gaps across 3T and reveal that LLM judges often conflate task capability and trust. A human study with 30 participants demonstrates the importance of the joint 3T optimization: participants show sharp trust declines after intervention misalignment despite correct outcomes, and prefer sleep-time assistance, even when imperfect, to preserve ongoing focus. Together, these findings support designing and evaluating proactive agents through the joint consideration of useful work, compute allocation, and evolving user trust.