Parameter schemas strongly influence tool choices in AI systems
Which Metadata Matters for LLM Tool Selection in MCP? Evidence from Controlled Factor-Level Interventions
Artificial IntelligenceComputation and LanguageSoftware Engineering
Summary
When AI language models pick tools to help answer questions, they rely on short descriptions and details about how to use the tool rather than the tool’s inner workings. The authors found that even small differences in how these usage instructions are written can change which tool the AI chooses, even if the tools do the same thing. Names of the tools mattered less than how their input parameters were described. This means that careful design of these instructions can steer AI to pick certain tools over others.
What this means in practice
- •For ai system developers: Design parameter schemas thoughtfully to guide large language models toward preferred tools in complex AI systems.
- •For software integration teams: Optimize tool metadata and ordering to influence which implementation an AI system selects without changing tool functionality.
Authors
Xiaofan Li, Fengchun Qiao, Xing Gao
Abstract
In Model Context Protocol (MCP) systems, LLMs select tools from metadata rather than from their implementations. Names and descriptions summarize what a tool does, while the parameter schema specifies the fields a tool accepts and how they are organized. Our analysis of metadata for 367,174 MCP tools reveals substantial schema variation among functionally similar tools. Many of these tools share names and similar descriptions but expose different parameter schemas. This raises a key question: can parameter-schema differences influence tool selection before any call is made? We study this question through controlled metadata interventions across five LLMs and five tasks. Each comparison changes exactly one metadata factor (server name, tool name, description, or parameter schema) while keeping the others fixed. We balance tool positions and use conditional logistic regression to estimate position-adjusted effects. We find that: (1) parameter-schema changes produce the largest and most consistent selection shifts among the tested interventions. Models tend to favor separate top-level parameters over grouped or composite forms. (2) Descriptions that explain parameters also tend to increase selection, while name effects are weaker and less consistent. (3) Targeted comparisons show that schema effects depend on the specific parameter names and on how the request represents the same information. When only one schema satisfies an explicit request requirement, all five models select the compatible tool despite position preferences. These results show that parameter schemas influence not only how tools are called, but also whether they are selected. For MCP developers and evaluators, schema design and tool ordering can therefore affect which implementation is invoked even when functionality is unchanged.