Waspmem helps ai agents improve complex procedures with human diagnosis

WASP-Mem: Let Agent Organize Procedural Memory. Human Spend Effort on Diagnosis

Artificial IntelligenceComputation and LanguageMachine Learning

Summary

Automating complicated workplace procedures is hard for AI agents because they can’t anticipate special company rules or unexpected problems during the work. The authors introduce WASP-Mem, which lets agents store and organize what they learn from experience in linked wiki pages without needing extra training. This helps agents get better at following procedures by learning from both successes and failures. Their study found that letting agents handle organization while humans focus on diagnosing failures leads to the best and most efficient improvements.

What this means in practice

  • For enterprise automation teams: Improve automated workflows by having agents consolidate experience into wiki-style procedural memory while focusing human effort on diagnosing failures to boost recovery and success rates.
  • For customer support engineers: Use agent-organized procedural memory combined with human diagnostic input to handle complex troubleshooting paths that are undocumented or state-dependent.

Authors

Peizhi Niu, Ge Shi, Vaibhav Gorde, Sarthak Ghosh, Christos Faloutsos, Min Cheng, Kevin Chen

Abstract

LLM agents struggle to automate complex enterprise procedures because static, pre-built harness cannot capture proprietary policies, undocumented behaviors, or state-dependent recovery paths that surface only during execution. Procedural memory accrued from agent experience offers a training-free path to closing this gap -- yet unlike episodic records or semantic facts, it must consolidate repeated successes and diagnosed failures into reusable actions and recovery strategies. We introduce WASP-Mem (Wiki-based Agent-Synthesized Procedural Memory), a nimble but effective method that distills execution traces and feedback into persistent, interlinked wiki pages, enabling continual agent learning. On TheAgentCompany benchmark, WASP-Mem raises the checkpoint pass rate by 4.6-6.1% over the strongest baseline we compared across three models while requiring 57-61% less storage. We further present a systematic controlled study spanning consolidation method, feedback type, trace ingestion scope, and refinement rounds. Our results reveal that prescriptive knowledge organization such as human defined schemas and explicit deduplication rules hurts performance rather than closing the procedural-knowledge gap. In contrast, human diagnosis converts failed traces into corrective knowledge and drives sustained improvement: a single round of human diagnosis outperforms four rounds of self-judgment by 7.0-10.9%. These findings point to a simple division of labor: let agents organize procedural memory, and invest human effort where it matters most -- diagnosis.