Self-evolving memory improves teamwork in long tasks for ai agents
SEPM: Self-Evolving Procedural Memory Grounded in Task Graphs for Multi-Agent Collaboration
Multiagent Systems
Summary
Long tasks that need many agents working together require them to remember not only what actions worked, but also how they coordinated. The authors propose SEPM, a method that links agents’ memory with the tasks’ structure to help them share and update plans smoothly. SEPM tracks what happens during tasks and learns from successes and failures to build routines that keep evolving. This helps agents work together better and reuse what they learned in new team tasks. Tests show SEPM improves performance compared to other memory methods in various simulated teamwork environments.
What this means in practice
- •For multi-agent system developers: Improve coordination and memory sharing in AI agents working on complex, multi-step tasks for better team performance.
- •For robotics software engineers: Enable robot teams to evolve and reuse collaborative task plans dynamically to handle long and varying jobs with less manual programming.
Authors
Yi Liu, Junwei Zhao, Li Sun, Jiansong Zhang, Sen Su
Abstract
Long-horizon multi-agent collaboration requires memory that preserves both successful actions and the coordination structure underlying them. Existing memory systems support information sharing or procedural reuse, but often leave the evolution of shared procedures disconnected from the task dependencies and execution states of collaborating agents. This coupling also limits the direct transfer of procedural memory designed for individual agents to team settings. We propose Self-Evolving Procedural Memory (SEPM), a framework that uses task dependency graphs as a common representation for online coordination and procedural memory evolution. During execution, a lightweight shared blackboard records structured evidence alongside task dependencies, supporting the detection of goal, plan, and world-state divergence and evidence-based realignment. After execution, verified actions and dependencies are distilled into reusable procedures, while recurring failures inform warnings and constraints. The shared procedure pool evolves through creation, refinement, specialization, and retirement, guided by success and cost utility and constrained by evidence and safety requirements. Together, these mechanisms preserve the collaborative structure and applicability conditions needed to reuse experience across tasks. Evaluations on ALFWorld, WebArena, and OfficeBench under AutoGen and DyLAN show that SEPM achieves the highest task scores among the compared methods in all six benchmark settings. Under AutoGen, it improves task scores over the no-added-memory baseline by 29.85, 25.86, and 13.00 percentage points, respectively. The codebase is available at https://github.com/cookieApril/SEPM.