Explicit responsibility boundaries improve safety in language model agents
Responsibility Boundaries for Agentic Systems: A Comparative Evaluation of Probabilistic Agentic Programming
Software EngineeringArtificial IntelligenceProgramming Languages
Summary
Language model agents can do many things like reasoning and using tools, but it’s hard to clearly say who or what is responsible for their actions. The authors explain a way called Probabilistic Agentic Programming (PAP) that makes these responsibility boundaries clear and checks that agents act safely. They tested PAP on different business tasks with many models and found it can reliably enforce rules and evidence for agent actions. This helps keep AI agents accountable and within limits when they perform tasks.
What this means in practice
- •For enterprise automation teams: Ensure AI-driven workflows respect predefined limits and approval steps to improve operational safety and traceability.
- •For software engineers building ai tools: Design AI agents with explicit responsibility contracts that enforce resource controls and evidence collection during tool use.
Authors
Khadim Lo, Soukaina Alaoui Ismaili, Jean-Sébastien Dessureault
Abstract
Language-model agents combine reasoning, tools, memory, delegation, and external effects. Authority is often distributed across prompts, host code, configuration, and logs. Probabilistic Agentic Programming (PAP) makes responsibility boundaries explicit programming abstractions connecting mandate, authority, effects, and evidence. We compare PAP with programming paradigms, agent-oriented and BDI systems, workflow and policy languages, and LLM tool-use frameworks along seven dimensions. A mediation kernel formalizes checks before tool execution, operation-bound approval, resource limits, and evidence. Inductive proofs establish conditional safety for arbitrary model responses, assuming correct guards and complete mediation. A backward simulation matches graph executions to source executions in a defined sequential Touba subset. Its human checkpoints release program continuations; operation-bound approval remains a separate kernel contract. Finite-state exploration checks the model. Scripted enforcement tests exercise selected refusal paths and permitted controls. An exploratory study combines seven business cases, including two three-agent workflows, seven models from three providers, five methods, and five repetitions per case and method. Shared English instructions and exact checks of stored outcomes yield 1,225 assessed observations. Native Touba completes all 35 observations on six models and 30 on the remaining model; structured Touba ranges from 21 to 35. Native matches the highest observed count for each model, including ties with the frameworks. All paired case intervals include zero. These observed counts describe the evaluated tasks; the runtime authority boundary remains fixed. The results connect responsibility contracts, execution rules, and evidence.