Papers for

ai platform security teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Attack method hijacks AI agents by manipulating tool metadata and outputs

A2M: Trace-Optimized Agent Hijacking in the MCP Ecosystem

Abstract: Agents using the Model Context Protocol (MCP) rely on semantic matching to select tools from third-party servers, exposing a semantic supply-chain risk through attacker-controlled metadata and outputs. We introduce A2M (Attraction-to-Manipulation), a two-stage black-box framework for hijacking MCP agents. The Attraction phase optimizes tool metadata to increase invocation probability; the Manipulation phase uses execution traces to refine adversarial tool returns that steer agents toward attacker-desired outcomes. On LiveMCPBench, direct attacks optimized and evaluated on GLM-4.6 achieve a macro-average malicious tool invocation rate of 93.6% across four scenarios, increase weighted token costs to 32.4$\times$ the benign baseline under Cognitive Denial of Service, and attain a mean attack success rate of 74.4% across Information Exfiltration, Environment Integrity Compromise, and Reasoning Derailment. Transfer to four other models without re-optimization yields corresponding macro-averages of 63.6%, 2.7$\times$, and 24.5%. These findings motivate stronger tool vetting and runtime isolation in MCP ecosystems. Code is publicly available at https://github.com/Lilaizhen/A2M.

Tue 22 SeptCryptography and SecurityArtificial Intelligence
The gist
Some AI agents pick tools to do tasks based on descriptions they receive, but this makes them vulnerable if those descriptions are controlled by attackers. The authors present a two-step attack that first makes a malicious tool more likely to be chosen by the agent and then tricks the agent during execution to produce harmful behavior. They tested this attack on several AI models and scenarios, showing high success rates in making agents call malicious tools and misbehave. This highlights the need to better check tools and isolate them during use to prevent such attacks.
Open 2609.26761v1