MemCodex improves language agent memory with evolving hierarchical programs
MemCodex: Self-Programming Hierarchical Memory for Language Agents
Computation and Language
Summary
Language agents often need to remember and connect different pieces of information to answer questions. The authors created MemCodex, a memory system that organizes information into layers and evolves its own ways to store, find, and combine memories efficiently. Instead of fixed memory rules, MemCodex adapts by rewriting its processes and stops searching as soon as it has enough evidence. This approach helps agents answer tasks more accurately and faster while needing less memory space.
What this means in practice
- •For chatbot developers: Build language agents that adaptively manage memory for faster and more accurate multi-step question answering.
- •For automated customer support teams: Employ adaptive memory systems to improve response quality and speed by dynamically retrieving relevant past interactions.
Authors
Xiaoqiang Wang, Bang Liu
Abstract
Agent memory faces heterogeneous access needs: a single-hop question may require one piece of evidence, whereas a multi-hop question must combine evidence from multiple sources. Predefined memory workflows cannot adapt to these varying needs. Recent adaptive methods search or learn over memory components and their compositions, but the design space itself remains predefined. We introduce MemCodex, a self-evolving hierarchical memory system that organizes experience into executable memory programs for summaries, relational knowledge, reusable skills, and latent memory. Open-ended program evolution searches the open design space of layer programs by rewriting how each layer is constructed, indexed, retrieved, and routed, thereby adapting both within-layer implementations and cross-layer composition. At query time, reads traverse the hierarchy from coarse to fine and stop once sufficient evidence is found, descending to the original history when needed. We further develop MemArena, a unified runtime that places heterogeneous data and memory systems behind a common interface. MemCodex improves average task success by 10.1% relative to the strongest adaptive-memory baseline, while using 3.4x fewer context tokens and achieving 2.1x faster inference.