Agentic web technologies baseline shows task specific performance gains
A Baseline Assessment for the Agentic Web Technology Stack
Artificial IntelligenceEmerging TechnologiesSoftware Engineering
Summary
There are many new tools and rules for making AI agents that can work together on the web, but it’s confusing to know which ones work best. The authors created a system to test these AI tools together on 300 different tasks. They found that picking the right combination of AI models, communication methods, and software frameworks helps complete tasks faster, with less extra talk and more stable results. This work gives a starting point for building and comparing complex AI systems that work together online.
What this means in practice
- •For multi-agent system developers: Choose agent technology components to improve AI task success, speed, and communication efficiency based on benchmarked results.
- •For ai infrastructure architects: Plan governance and infrastructure needs by understanding combined effects of protocols and frameworks on multi-agent web solutions.
- •For customer service platform teams: Build multi-agent AI assistants tailored to specific tasks that complete requests more accurately and efficiently.$Commercial implications: Enables development of commercially viable multi-agent AI assistants that outperform general solutions in task handling for customers.
Authors
Vagner Figueredo de Santana, Diogo Cortiz
Abstract
The fast pace in which protocols, frameworks, and technologies are being proposed for LLM-based agents results in a complex scenario for anyone aiming to create state-of-the-art AI solutions. Without an assessment on how different protocols, frameworks, and libraries work together to compose an Agentic Web solution, developers and organizations face different challenges (e.g., skilling, governance, infrastructure) even before becoming part of the Agentic Web ecosystem. In this context, we propose a system for performing Agentic Web evaluations called MASTIF (Multi-Agent System TestIng Framework) and present a baseline assessment of Agentic Web technologies, including 3 LLMs, 3 protocols, and 4 agent implementation frameworks, against 300 Mind2Web benchmark tasks. Results suggest that selecting a task-specific technology stack may result in gains of task completion rate, efficiency score, and social value stability scores; and reductions in verbosity and time to complete tasks. With the presented results, we expect to present a baseline in terms of metrics and results for this complex and fast evolving scenario for those aiming at designing, assessing, and developing multi-agent systems.