Specifying the Delegated-Autonomy Boundary: Requirements Engineering for Agentic AI
Software EngineeringArtificial Intelligence
Summary
The authors explain that AI systems which can make their own decisions and act on them—called agentic AI—need clear rules about what they are allowed to do and when. They introduce the idea of a 'delegated-autonomy boundary,' which means deciding how much control the AI should have and under what circumstances. To manage this, the authors propose two tools: one helps decide when to use such AI, and the other sets detailed rules on the AI’s role, authority level, and oversight. They show how this works with examples like a hospital agent and a code review agent.
Authors
Chetan Arora, Andreas Vogelsang, Abbi Sharma
Abstract
Agentic AI systems do not just predict or recommend; they plan, maintain state, and act in external environments with varying degrees of autonomy. This changes the requirements engineering problem in a specific and under-addressed way: it introduces what we call the delegated-autonomy boundary -- the set of decisions about what may be delegated to the system, under what graduated authority, with what oversight, and how control is returned. Current practices bury these decisions inside prompts, tool schemas, and runtime policies, even though they are requirements-level commitments. This paper proposes two complementary artifacts. First, an Agency Justification Record (AJR) helps teams decide when an agent is warranted over simpler alternatives. Second, an Agentic Delegation Policy (ADP) captures what must be specified for safe and effective development: purpose, authority, information, coordination, assurance, and evolution. Crucially, authority in the ADP is modelled as graduated, i.e., a tiered structure. We illustrate the framework with two contrasting examples: a safety-critical hospital discharge coordination agent and an automated code review agent.