Persistent AI assistants learn when and how to help users best
When Intelligence Becomes Agency: A Theory of Governed, Proactive Agency for Symbiotic AI Systems
Artificial Intelligence
Summary
Many AI helpers today only act when told, but truly useful assistants need to decide when it's right to act on their own. The authors explore how AI can be designed to choose when and how to help users based on ongoing understanding, permission, and user preferences. They propose a framework that links AI decisions to clear rules and limits set by the user, ensuring the AI respects authority while being proactive. This approach aims to make AI assistants more reliable and trustworthy by keeping them accountable and adaptable over time.
AI assistantsproactive agencyautonomous behaviordelegationactivation problemauthorizationaccountabilityhuman-AI interactionbehavioral episodespersonalization
Authors
João Dias Ferreira
Abstract
Persistent AI assistants are intended to extend human attention, memory, and coordination across changing digital and physical environments. To be truly useful they must do more than just act when asked. They must decide on their own whether a situation warrants behavior at all, when it does and in what mode, whether to act, ask, monitor, defer or deliberately refrain. We call this the activation problem. Research on commitment, appraisal, mixed-initiative interaction and delegation each illuminates part of it, but none ties situated activation to continuing authorization and accountability. This paper develops a conceptual and formal framework for governed proactive agency, organizing behavior across time through perception, intent, affective-conative appraisal, constraint, and feedback. It distinguishes autonomous and delegated agency and defines symbiotic agency as delegation under a standing, revocable mandate, with continuing coupling to the principal's situation, calibrated inference of their condition, and bounded personalization. The distinctive contribution is an integrated account linking activation decisions to authorized perception, behavior selection, authority containment, traceable restraint, and constrained adaptation, with behavioral episodes as the unit of analysis. Through an agency classification method, an evaluation framework, proposed benchmark scenarios, and a reference architecture, the account provides a basis for specifying and assessing whether assistance is warranted, timely, authorized, and answerable beyond task completion alone. It is intended to guide the development and evaluation of always-present personal assistants and embodied support systems that augment human capabilities while preserving the principal's authority and judgment.