MCP gateways shape AI agent tool use and security performance
GatewayBench: Unified Capability and Security Evaluation Platform for MCP Gateways
Artificial Intelligence
Summary
AI agents use many tools to interact with the world, and MCP gateways act as middlemen that manage these tools. The authors studied different ways these gateways are designed to see how they affect what agents can do, how well they do it, and how safely. They created GatewayBench, a big testing system with many real tools and tasks, to compare various gateways. Their experiments show that gateway designs make a big difference in the agent’s success, efficiency, and defense against attacks.
What this means in practice
- •For ai system builders: Evaluate and compare MCP gateway designs to improve AI agent task success, efficiency, and security under realistic tool conditions.
- •For cloud service operators: Test gateway configurations to enhance the security and governance of large-scale AI agent environments accessing diverse tools.
Authors
Yitian Zheng, Zhaorun Chen, Haoyu Li, Bo Li
Abstract
AI agents have increasingly interacted with external environments by invoking tools, with Model Context Protocol (MCP) emerging as a standard interface for connecting agents to heterogeneous tools and services. As organizations deploy agents at scale, MCP gateways have become a critical middleware layer for aggregating, governing, and shaping the tool surface exposed to agents. Despite the growing importance of MCP gateways in agentic systems, no prior work has systematically studied how gateway design affects agent capability, efficiency, and security. To address this gap, we introduce GatewayBench, the first unified evaluation framework and benchmark for MCP gateways. We characterize gateway architectures into two major paradigms: flattened gateways and hierarchical gateways. GatewayBench comprises 524 tasks across six domains, together with a pool of 11,118 real-world tools from 251 authentic MCP servers. To emulate large-scale deployments, we systematically inject task-irrelevant tools sampled from this pool into each agent environment, enabling controlled evaluation as the accessible tool surface scales. Based on GatewayBench, we conduct extensive experiments across seven widely deployed MCP gateways and diverse agent configurations, covering both benign and adversarial tasks. Our results reveal substantial differences across gateway designs in task completion, tool-use efficiency, and resilience to attacks, and further demonstrate that these effects interact strongly with the underlying model and agent harness. These findings highlight the MCP gateway as a consequential yet underexplored component of the agentic AI stack. By enabling controlled evaluation of gateway architectures under realistic tool ecosystems, GatewayBench provides both practical guidance for MCP gateway design and a rigorous testbed for developing more capable, efficient, and secure agentic systems.