Graphmemory cuts memory use for adapting language model context
Decoupling Memory from Context: Structured Memory for Token-Efficient Test-Time Continual Learning
Artificial IntelligenceMachine Learning
Summary
Managing memory in large language models is tricky because the more information you add, the slower and less accurate they can become. The authors came up with GraphMemory, a system that organizes relevant information like a graph and only shows the important pieces to the model when it needs them. This method helps the model adapt quickly without slowing down or using too much memory. Experiments show that GraphMemory does as well as other methods but uses much less memory.
What this means in practice
- •For enterprise ai teams: Enhance domain-specific language model agents to adapt online more efficiently by using structured memory without increasing input size.
- •For medical data engineers: Support language models that accumulate and refine medical knowledge over time while keeping inference costs manageable.
Authors
Yehya Farhat, Michael Desmond, Anastasios Kyrillidis
Abstract
Large language models (LLMs) are increasingly deployed in enterprise, scientific, and medical applications, where agents must incorporate domain-specific knowledge and adapt from experience. Context engineering offers a practical alternative to weight updates by improving model behavior through instructions, strategies, and evidence supplied at inference time. However, adapting context online typically requires a costly trial-and-error process, while queries are often processed independently, preventing useful experience from carrying forward. Memory systems address this limitation by retaining information across interactions, but approaches that continually append information to a shared context face increasing token costs, context-window limits, and performance degradation as the context expands. We introduce a unified formulation of context optimization and show that an agent memory system update can be interpreted as an optimization update procedure over the model's context. This perspective attempts to provide a principled framework for studying memory design and its efficiency. We then propose GraphMemory, a lightweight graph-based memory that accumulates, refines, organizes, and connects reusable strategies. For each query, GraphMemory retrieves only the relevant subgraph, enabling online context adaptation without exposing the model to the entire memory. Under bounded retrieval, the amount of retrieved memory remains constant as the number of processed examples grows. Experiments show that GraphMemory achieves competitive downstream performance while using approximately 81-85% fewer memory-construction tokens than our baselines.