Graph methods compared for recalling long term agent conversations
TAGGRAPH: Tag-Augmented Graphs for Graph Retrieval of Agent Persistent Histories
Information RetrievalArtificial Intelligence
Summary
Remembering past conversations helps AI agents stay consistent, but it's hard to compare different memory methods because they use different ways to store and find information. The authors tested several graph-based and text-based retrieval methods on shared conversation records to see which recalled information best. Some graph methods worked well, but a text-based method called BM25 and an input-based method named OpenClaw often performed better. They found that how information is tagged and organized heavily affects how well the system retrieves memories.
What this means in practice
- •For chatbot developers: Improve chatbots by comparing memory retrieval methods to keep agents consistent across conversations using shared conversation history graphs.
- •For digital assistant teams: Choose better memory retrieval setups for assistants by evaluating graph and text-based methods on consistent user interaction histories.
Authors
Yu-Su Chen, Yu-Jung Liang, Pengtao Xie
Abstract
Long-term memory lets LLM agents recall past interactions and remain consistent across sessions, but memory systems are hard to compare because they often vary in representation, indexing, retrieval, and evaluation. We present a controlled evaluation framework based on shared 5W-style conversational memories. Localized graph configurations traverse a common base graph; AdaptiveGraph adds chronological edges and Personalized PageRank diffusion. We also evaluate BM25 over the same extracted notes and OpenClaw as a raw-input external reference. Retrieval rankings vary across memory settings. On LongMemEval-S, AdaptiveGraph is the strongest graph configuration at 0.844 MRR, but BM25 reaches 0.867 and OpenClaw 0.880. On ATANT Core, localized graph traversal outperforms diffusion and BM25, whereas BM25 leads the stress rounds. Reducing LongMemEval-S within the tested range does not reproduce the ATANT diffusion penalty, but the smallest tested store remains larger than ATANT Core, so store size cannot be ruled out. The penalty also persists under a permissive content-match criterion. Vocabulary normalization and extraction quality substantially affect graph retrieval, and missing extraction tags are common among top-five misses. Retrieval strategies should therefore be evaluated jointly with the memory setting and against strong lexical baselines.