Multi agent llms show distinct trust gaps in communication and actions
Who Speaks, Who Authorizes, Who Acts? Understanding Trust Boundaries in Multi-Agent LLMs
Artificial Intelligence
Summary
Large language models (LLMs) working together send messages to decide on actions, but there are different steps where trust can break down: who sends the message, who has the authority to approve it, and who actually does the action. The authors studied how these trust boundaries interact and found that simply showing the message source does not always help, tools may refuse harmful requests but models sometimes still ask for them, and proofs of authorization don’t always mean the claim is true. These findings suggest it’s important to separately check the information sent, the authorization given, and the action taken.
What this means in practice
- •For security engineers: Improve security checks in multi-agent AI systems by separately verifying message origin, authorization proofs, and actual tool actions.
- •For ai system integrators: Design multi-agent LLM workflows that recognize trust gaps between message passing, authorization, and execution to reduce false-action risks.
Authors
Junru Zhu, Xin Chen, Hang Lyu, Jiayue Meng, Muzi Li, Yifan Chen, Hengyi Huang, Yu Sun, Weiwei Qi
Abstract
Multi-agent LLM systems exchange messages before acting through tools, but message delivery, signing authority, and tool-side enforcement are separate trust boundaries. We represent them as communication, authorization, and execution structures, and use the authority cut to characterize the smallest independently compromised signer coalition that can construct a policy-valid false-claim proof under stated assumptions. Controlled paired hidden-state studies reveal four distinctions. First, no fixed-message routing contrast across four models meets the prespecified large-effect criterion; when relays rewrite messages, executed false-state action rates vary by model and route configuration. Second, presented source-identity cues do not uniformly increase protected-action selection across three models. Third, on invalid-proof cases, Nemotron and Llama request protected actions in 52.1% and 54.2% of trials, respectively, yet a deterministic tool gate commits none: request-level and commitment-level safety readouts diverge. Finally, a separate single-model authorization check shows that a false claim with a policy-valid compromised-signer proof can pass the gate, since proof validity is not factual truth. These findings motivate evaluating delivered information, qualified authorization, model proposals, and committed actions separately.