Textual memory updates reduce error persistence in long tasks

KEEPER: Learning Control-Sufficient Textual Memory for Long-Horizon LLM Agents

Computation and LanguageArtificial Intelligence

Summary

Keeping track of information over a long task is tricky because early mistakes can stick around and cause problems later. The authors studied different ways to help a computer remember what it learned in a way that avoids repeating errors. They found that focusing on rewards from past actions helped the system learn better and avoid mistakes compared to other methods. Their work shows that it’s important to check not just how well a task is done but also whether the system is holding on to correct information.

What this means in practice

  • For ai developers: Improve memory reliability in language-based agents handling tasks with many steps by reducing error carryover in their textual memory updates.
  • For chatbot engineers: Enhance the accuracy of chatbots performing lengthy interactions by incorporating reward-based memory supervision to minimize incorrect claim persistence.

Authors

Jinkyeong Choi, Chaebin Jeong, Junsung Jeon, Donghyeon Park

Abstract

Recursive textual memory can carry earlier errors into later updates, yet task performance alone does not establish memory reliability. We define self-consistency bias as the persistence of incorrect claims already present in the incoming memory. We present KEEPER, a comparative analysis of four auxiliary objectives along information-content and update-sensitivity axes within a shared recursive interface, examining how supervision targets relate to agent performance. Among evaluated conditions, reward-related prediction achieve the highest task performance before and after offline reinforcement learning, outperforming joint supervision. A claim-level analysis of one baseline trajectory documents error persistence despite corrective observations. Together, these analyses motivate evaluating error persistence and valid-information retention alongside task outcomes.