Align agent responsibilities and powers to avoid system conflicts
Aligning Institutional Power and Operational Ability
Artificial IntelligenceMultiagent Systems
Summary
AI systems with multiple agents need to make sure that the responsibilities assigned to each agent match what they can actually do. If an agent is expected to do something impossible or can unfairly gain extra power, problems can arise. The authors propose a way to describe and analyze these systems using models of processes and institutional power. This helps detect and fix mismatches between what agents are supposed to do and what they can do, ensuring more socially responsible AI behavior.
What this means in practice
- •For multi-agent system designers: Analyze and adjust agent responsibilities to avoid unsatisfiable expectations in complex AI systems.
- •For organizational compliance teams: Use process-oriented models to identify where institutional power imbalances create policy enforcement issues.
A position paper. It proposes an approach and reports no results.
Authors
Andrea Marino, Giovanni Sileno, Thomas van Binsbergen, Tom van Engers
Abstract
To be socially sustainable, Agentic AI systems require normative designs that align responsibilities with the ability to fulfil them. Identifying incongruences such as unsatisfiable expectations or privilege escalation is crucial in avoiding normative pitfalls. In this study, we discuss design dimensions suitable for describing multi-agent systems in light of this norm-ability alignment requirement. We propose a formal framework centred on process-oriented modelling and institutional power for the analysis of a given normatively relevant process, discuss strategies for repairing it, and elaborate on how the symmetrical relation between operational process and institutional state affects the lifecycle of normative positions.