MetaExp teaches language models to improve themselves across tasks
MetaExp: Learning to Self-Improve through Experience via Verifiable Cross-Task Environment Scaling
Artificial Intelligence
Summary
Language models that interact with environments learn from feedback, but turning that feedback into reusable experience is hard. The authors introduce MetaExp, a method that trains models to learn from many related tasks by focusing on shared tools or workflows, helping them gather useful knowledge beyond single tasks. They show that MetaExp improves model performance on various benchmarks and that these improvements transfer even to new domains. This suggests experience learning can become a general skill that helps models keep improving after deployment.
What this means in practice
- •For ai development teams: Enhance language model agents to transfer experience across varied tasks for more efficient learning after deployment.
- •For automation engineers: Create smarter automation systems that improve their performance by learning reused procedures and tools from multiple related tasks.
Authors
Renhao Li, Xinjie Shen, Yang Su, Jianhong Tu, Derek F. Wong, Dayiheng Liu, Min Yang
Abstract
Language-model agents can accumulate interaction feedback, but reliably turning that feedback into reusable experience remains challenging. Existing methods either optimize external experience scaffolds or train experience-related behavior on specific task distributions, leaving open whether experience learning itself can become a transferable model capability. Training such a capability calls for supervision beyond isolated tasks, which mainly support in-episode problem solving rather than revealing what knowledge remains useful across episodes. In this work, we introduce MetaExp, a scalable framework that trains this capability through verifiable task families with controlled cross-task structure. Each family preserves a reusable factor while varying task-specific realizations, enabling transfer without direct solution reuse. We propose two complementary designs: Shared-Tool families keep black-box tools fixed across tasks to elicit operational experience, while Shared-Workflow families preserve latent procedures across changing realizations to elicit procedural experience. We scale environments under both designs, collect and filter sequential interaction trajectories, and fine-tune Qwen3.5-122B-A10B on 4,956 trajectories from 1,000 task families. With parameters frozen during downstream evolution, MetaExp improves by +8.5 points on SpreadsheetBench, peaks at +2.3 on APEX-Agents, and improves by +14.8 on AutomationBench. The gains persist under domain-excluded training, supporting experience learning as a transferable capability for continued non-parametric improvement after deployment.