Authoritylens measures real power balance in llm agent systems
AuthorityLens: Rethinking LLM-Based Agent Systems Through the Lens of Authority
Artificial Intelligence
Summary
Large language model (LLM) agents are being used to make important decisions and handle valuable resources, often alongside humans or other AI agents. The authors noticed that the official permissions and roles don’t always match who actually has control in practice. They created AuthorityLens, a new way to measure authority by looking at what a system can do, who needs to cooperate to get it done, and how power is shared among participants. By testing this on several popular LLM systems, they found surprising differences between intended and actual authority. This tool helps make sense of real power structures in complex AI workflows.
What this means in practice
- •For software engineers: Design more trustworthy AI systems by analyzing and balancing real authority among agents and humans using AuthorityLens.
- •For security teams: Improve oversight of AI-driven operations by detecting unintended concentration or separation of authority in agent workflows.
Authors
Shaojin Chen, Huihao Jing, Wun Yu Chan, Wenbin Hu, Jiaxing Li, Wu Pandy Pui Ching, Kshitij Bhatia, Xinlei He, Haoran Li, Yangqiu Song
Abstract
LLM-based agents are increasingly deployed with authority over consequential resources and decisions in real systems. These agents often operate alongside human and LLM-based participants who hold different forms of authority. Yet workflow roles, permission settings, and review mechanisms do not necessarily reflect the authority realized in practice. We introduce AuthorityLens, a framework for measuring a system's authority structure. Starting from an authority portfolio, we evaluate a system along three dimensions: what the system is authorized to do (System Authority), how much joint participation is required to exercise that authority (Authority Separation), and how much authority each participant holds (Principal Authority). We derive these measurements from the minimal combinations of participants sufficient to realize each outcome across admissible runtime states. We apply AuthorityLens to Codex, OpenCode, and Gemini CLI across 13 operating configurations over a common portfolio of agent operations. We find that nominal configurations do not map cleanly onto realized authority. In Codex, Full Access changes System Authority only marginally while substantially concentrating authority in the executing Assistant. OpenCode's Build and Plan configurations have the same System Authority and Authority Separation despite different workflows and root-level permissions. In Gemini CLI, model-based review increases Authority Separation without changing System Authority. Principal Authority further distinguishes authority replication from authority separation: spawned or delegated agents can become alternative holders of the same authority without increasing the required joint participation. Together, these results demonstrate that AuthorityLens provides a unified framework for measuring and comparing realized authority structures across agent systems.