Evolving data generation boosts interactive agent learning success
Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning
Artificial IntelligenceMultiagent Systems
Summary
Improving how AI agents learn from experience often means creating helpful fake data. The authors address a problem where current methods don’t clearly guide how to make better fake data when the AI struggles. They introduce EvoSyn, a system that uses feedback from the learner to actively adjust and expand what kinds of experiences it creates, focusing on areas where the AI needs more practice. Tests show EvoSyn helps agents succeed more often and with richer training examples.
What this means in practice
- •For ai system developers: Create interactive AI training environments that automatically refine training data based on the AI’s learning gaps.
- •For simulation platform engineers: Build simulation tools that generate more effective synthetic experiences by evolving scenario parameters from agent feedback.
Authors
Wei Yang, Qing Ping, Youzhi Luo, Yawei Wang, Abhishek Kumar, Panpan Xu, Jun Huan
Abstract
The continued improvement of large language models increasingly depends on synthetic experience, yet effective data synthesis for interactive agents requires not only executable and verifiable trajectories, but also interactions that provide learning value for the current task agent. Existing methods increasingly incorporate environment feedback, data validation, or learner failures, but still face a feedback-to-decision gap: observed weaknesses do not directly specify how future synthesis should change, or whether they can be represented by the current synthesis space. To address this gap, we propose EvoSyn, a learning-feedback-driven multi-agent framework that organizes synthesis around a revisable state specifying what interaction factors can be deliberately generated and where synthesis effort should be allocated. A role-specialized multi-agent system translates learner feedback into synthesis revisions, while a Designer allocates synthesis directions and a Planner grounds them into executable experience. This process enables within-space adaptation toward weak or under-covered regions and synthesis-space expansion that promotes newly discovered factors into explicit generation controls. Experiments on $τ^2$-Bench and CAR-Bench show that EvoSyn improves average task success by 13.3 points over the strongest synthesis baseline, while producing higher-quality and structurally broader synthetic experience and discovering controllable factors missed without learner feedback.