MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation
2026-06-01 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors explain that the Model Context Protocol (MCP) helps connect large language models with external tools, but current tests do not show how well these models handle apps that use personal data. To address this, they created MCP-Persona, a new test set made up of popular social and work apps that rely on personal accounts. Their tests show that top AI agents have a hard time using these personalized tools correctly. This new benchmark helps uncover these shortcomings and guide future improvements.
Model Context Protocollarge language modelsbenchmarkspersonalized toolssocial media platformsenterprise collaborationagent performanceRedditSlackMCP-Persona
Authors
Wenhao Wang, Peizhi Niu, Gongyi Zou, Xiyuan Yang, Jingxing Wang, Haoting Shi, Yaxin Du, Jingyi Chai, Xianghe Pang, Shuo Tang, Yanfeng Wang, Siheng Chen
Abstract
The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social applications, where tools interact with individual accounts or local databases. To bridge this critical gap, we introduce MCP-Persona, the first benchmark specifically designed for evaluating agent performance on real-world, personalized MCP tools. MCP-Persona encompasses a diverse set of widely-used applications, ranging from social media platforms like Reddit and Xiaohongshu (Rednote) to enterprise collaboration suites such as Lark (Feishu) and Slack. Our extensive experiments on various state-of-the-art (SOTA) agents demonstrate their significant struggles with personalized tool use, thereby highlighting the benchmark's crucial role in identifying and addressing these limitations. MCP-Persona is publicly available at https://github.com/wwh0411/MCP-Persona}{https://github.com/wwh0411/MCP-Persona.