Evidence linked memory improves reliability of LLM agents

Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents

Artificial IntelligenceMultiagent Systems

Summary

Large language model (LLM) agents learn from past experiences stored in memory, but sometimes this memory can mislead them by providing incorrect or confusing lessons. The paper introduces WikiTrace, which links each learning lesson to the original detailed experience that generated it. This connection lets the agent double-check lessons and avoid mistakes when tackling new tasks. Tests show that using WikiTrace helps agents perform better and avoid errors caused by faulty memory.

What this means in practice

  • For ai system developers: Build more reliable LLM-based agents that verify lessons against original task details to reduce errors caused by faulty memory.
  • For customer support automation teams: Improve conversational agents by linking their learned responses to prior successful interactions, helping avoid repeating past mistakes.

Authors

Yukyeong Lee, Yumin Choi, Jinheon Baek, Sung Ju Hwang

Abstract

External memory enables LLM agents to improve from experience by distilling trajectories of past tasks into concise lessons for new tasks. However, memory can also lead to memory-induced degradation, where an agent fails on tasks it would solve without memory, often because it follows a lesson that is incorrect, or misapplies or misreads a valid one. This is because lessons are inferred from trajectories labeled only with task success or failure, yet are typically provided without the trajectories they came from, leaving the agent no way to check them. To address this, we introduce WikiTrace, an evidence-linked memory system designed to let agents check each lesson against its original trajectory. Specifically, motivated by the finding that a trajectory often still holds the evidence needed to catch an erroneous lesson, WikiTrace stores each lesson as a concise insight linked to a summary of why its trajectory succeeded or failed and to the raw trajectory itself. Then, on a new task, the agent starts from all insights and, whenever one lacks details needed to act, conflicts with another, or comes from a failed task, descends through its links with terminal tools to fill in missing details or correct the insight before acting. We validate WikiTrace on five agentic benchmarks, showing that it consistently outperforms existing memory approaches and a baseline given the same insights alone, while also reducing memory-induced degradation. Our code is available at https://github.com/yukyeongleee/wikitrace