CoMem improves large language models by learning from past failures
CoMem: Learning to Anticipate Failures from Retrieved Experience
Computation and Language
Summary
Large language models (LLMs) can learn from their mistakes, but usually only within a single task. The authors introduce CoMem, which helps these models remember and use experiences from many different tasks to better predict where they might fail. CoMem combines stored past examples with learned knowledge to improve problem-solving, especially in areas like math, coding, and complex question answering. This approach also helps smaller models provide useful guidance to bigger, more powerful solvers.
What this means in practice
- •For software developers: Use CoMem to improve debugging and code generation by learning from past code errors and solutions.
- •For customer support teams: Improve automated answer systems by making them learn from previous failures in handling diverse inquiries.
Authors
Yun Shen, Ming Shen, Tianjun Yao
Abstract
Reflection lets LLM agents improve from their own attempts, but the memory it builds is local to a single task, so an agent reflecting on its own tends to revisit the same explanation for failure. Cross-task reflective memory addresses this in two forms: retrieved memory keeps past experience explicit and looks up related cases, which is specific but bounded by the bank, and parametric memory internalizes experience into parameters, which generalizes but is grounded in nothing beyond the task description. When both are used, existing agents combine them additively, feeding their outputs to the solver side by side, so retrieved experience never changes what the learned memory produces. We introduce CoMem (Compounding Memory), which makes retrieved experience an input to the learned memory itself, at training and at inference. The agent builds a bank of its own successful and failed reflections, learns from retrieved cases to write failure hypotheses, candidate explanations of how an attempt is likely to go wrong, and at test time reasons from newly retrieved cases to guide each attempt. Varying the retrieved cases gives the memory different perspectives, yielding reflective diversity that is not merely sampled but also grounded in retrieved experience. Across two backbones and five benchmarks in math, code, and multi-hop QA, CoMem achieves strong overall performance. Additional evaluations show that it generalizes to out-of-distribution tasks and enables smaller models to provide effective guidance to stronger solvers.