Code mode cuts tool use cost and speeds agents without extra decision layer
Beyond Plain MCP: Do Decision Models and Code Mode Make Tool Use Cheaper and Faster?
Software EngineeringArtificial IntelligenceComputation and Language
Summary
Managing many tools for AI agents can be expensive and slow. The authors tested ways to make tool use cheaper and faster, comparing a simple method with one where agents write small scripts (code mode) and where a quick decision step picks tools. They found writing scripts saves lots of computing effort and improves results, especially when scripts include return format hints. Adding a decision step before script use did not help overall. The authors share their tools to help others test and build better AI helpers.
What this means in practice
- •For ai platform developers: Improve AI agent efficiency by replacing plain tool calls with code mode scripts including return format hints to reduce computational cost.
- •For cloud service operators: Optimize AI tool management by avoiding costly decision layers and focusing on streamlined script-based tool calls for scalability.
Authors
Abhishek Rai
Abstract
Model Context Protocol (MCP) gateways manage tool catalogs for language model agents using two primary paradigms: code mode where agents write scripts against tool stubs and decision layers which are cheap routers that preselect tool subsets. Using a controlled harness across 12 servers, up to 1,958 distractor tools, and three open-weight agents including Qwen, Gemma, and Nemotron, we present a preregistered evaluation on a sealed test split to determine when each approach succeeds. Code mode substantially outperforms plain MCP, cutting token usage by 70 to 90 percent while significantly boosting utility. Declaring return-shape hints in code mode stubs is a first-order variable, swinging success by up to 39.5 points for lower-performing agents, an effect larger than adding a router. The central hypothesis that a decision layer in front of code mode beats code mode alone is not confirmed, showing minor, statistically non-significant improvements for weaker agents and a slight cost for the strongest. We publicly release our harness, a finetuned 4B tool selection model, and all evaluation traces to support robust community experimentation on local AI privacy and agent architectures.