Model as data improves large language model agent harnesses across models
MaD-RSI: Model-as-Data for Recursive Self-Improvement of Agent Harnesses
Artificial Intelligence
Summary
Improving AI assistants often means fine-tuning the software that controls them, called a harness, for each individual large language model (LLM). The authors of this paper found that tuning these harnesses separately makes it hard to reuse improvements across different AI models and costs a lot in computing power. They suggest treating the differences between AI models as helpful clues during the tuning process, not just as a way to check results afterward. Their new method, MaD-RSI, uses these clues from multiple models to build a single, better harness that works well across many LLMs and requires less training time.
What this means in practice
- •For ai product developers: Create versatile control systems for AI assistants that perform well on different large language models without costly retraining.
- •For software infrastructure engineers: Reduce operational costs by training shared agent harnesses that adapt efficiently to multiple AI models’ diverse behaviors.
Authors
Yu Cui, Hong He, Ruiqing Yue, Sicheng Pan, Zhuoyu Sun, Yifei Liu, Ting Li, Wenzhuo Zhu, Baohan Huang, Haibin Zhang, Cong Zuo
Abstract
Recursive self-improvement (RSI) of agent harnesses has emerged as an effective approach to enhancing the capabilities of large language model (LLM) agents. By searching the harness space for a specific LLM on a given dataset, RSI can discover a harness tailored to the capabilities and behavioral characteristics of that model. However, such optimization often creates a strong coupling between the learned harness and its target LLM, limiting generalization across models and requiring substantial computational resources to adapt to new LLMs. Moreover, existing RSI methods typically optimize a harness using feedback from a single model and evaluate its generalization to other models only after training. This paradigm treats model differences primarily as a criterion for evaluating generalization rather than as valuable evidence for optimization. In this paper, we introduce a new perspective that treats models as data and places the harness at the center of optimization. Building on this perspective, we propose \textsc{MaD-RSI}, a framework that directly optimizes a shared harness across multiple heterogeneous LLMs by incorporating their diverse failure trajectories into the training process. Rather than assessing model differences only after optimization, \textsc{MaD-RSI} leverages these differences as training evidence to guide the self-improvement of a shared harness. Furthermore, we introduce a new evaluation framework with two complementary metrics designed to assess the effectiveness of harness optimization across models. Experiments across multiple datasets and LLMs demonstrate that \textsc{MaD-RSI} improves generalization over existing approaches while reducing training costs.