Dynamic agent systems improve task accuracy and efficiency per query
Dynamic Harness Search: Building Multi-Agent Systems Per-Query via Prediction
Machine Learning
Summary
Solving complex tasks with multiple AI agents usually requires setting up roles and tools carefully, which can be slow or need lots of trial and error. This paper presents SHIFT, a method that learns from past attempts to predict the best way to organize these agents for each new task, speeding up the process without testing many options each time. By using predictions, SHIFT builds customized setups that work better and use fewer resources than previous methods on thousands of tasks.
What this means in practice
- •For ai application developers: Create customized multi-agent workflows for diverse tasks that balance accuracy and computing cost more effectively than existing design methods.
- •For automation platform engineers: Integrate predictive models to automatically select the best agent setups per query, reducing manual design and runtime trial.
Authors
Som Sagar, Shasha Li, Hejie Cui, Ransalu Senanayake, Sercan Ö. Arık
Abstract
Agent harnesses specify the roles, instructions, tools, and communication structure used to solve a task, and the right harness depends on the query. Because the value of each design choice is observable only through execution, tailoring a harness to each query has required either executing alternatives at inference time or costly manual design. We introduce SHIFT, which moves execution out of the per-query search loop. A local LLM architect learns a policy over harness-building actions from search, and a value function that predicts, from measured executions, a utility balancing accuracy against execution cost. For each query, Monte Carlo tree search uses these predictions to construct a harness. Across 9,193 tasks in six benchmarks, from math to document and general-assistant tasks, with a Gemini 3.5 Flash executor, SHIFT attains the highest mean accuracy, about 80%, outperforming 17 baselines that span prompting, prompt optimization, and workflow search, and exceeding the strongest baseline by 7.2 percentage points. A cheaper mode of SHIFT also attains a higher mean accuracy than every baseline while using 32% fewer execution tokens than the strongest baseline. We further show that choosing structure, instructions, and tools jointly beats choosing only instructions or only tools by up to 9.1 percentage points, and that learned value selection identifies more accurate harnesses with lower execution cost from candidate pools.