Agentic system picks best model and harness for improved queries
HM-ROUTER: Joint Model and Harness Routing for Agentic Systems
Machine LearningComputation and Language
Summary
Choosing the right combination of a model and the way it is used—called a harness—is important for smart software called agents. The authors created HM-Router, a method that learns how to match models and harnesses for each question, even for pairings it hasn't seen before. This method shares knowledge across different models and harnesses to make better choices. Tests showed HM-Router picks the best pairs more accurately and works well even when many options were not tried during training.
What this means in practice
- •For software engineers: Build intelligent agents that automatically select compatible models and execution harnesses to improve task performance across diverse queries.
- •For ai platform operators: Optimize resource allocation by routing queries to the most suitable model and harness pair, including those not previously paired in training data.
Authors
Hao Mark Chen, Royson Lee, Yasuyuki Okoshi, Dimitris Anastasiou, Wayne Luk, Hongxiang Fan
Abstract
Agent performance depends on both the underlying model and the harness that manages its tool use and execution. Selecting a suitable pair requires accounting for their compatibility, yet training samples may cover only a subset of the growing combination space. We introduce HM-Router, a routing method that jointly selects a model and harness for each query. It learns separate model and harness representations shared across routes, with an interaction term inspired by canonical polyadic (CP) tensor decomposition to capture how their compatibility varies with the query. This sharing allows training samples from observed pairs to inform predictions for unobserved combinations. We curate a benchmark from 12 public agent benchmarks, covering 293 routes, 73 models, and 25 harnesses. HM-Router exceeds the strongest evaluated learned baseline by 7.3 percentage points in mean routing accuracy and leads at all seven evaluated cost budgets on the six-benchmark subset. When 90% of routes have their training outcomes withheld, allowing unobserved combinations improves normalized accuracy by 15.8 points over restricting the same router to observed routes. HM-Router has also demonstrated training sample efficiency for new routes and components and generalization to unseen benchmarks. Our code and data are open-sourced at https://github.com/hmarkc/HM-Router.