Mechanistic Attention Guidance for Agent Memory Refinement
2026-07-20 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors studied how computer agents remember and use information when making decisions. Most current systems update memory based only on written outputs, missing important hidden signals about how memory is really used during tasks. They found a way to track attention signals to see which memory parts are important, then used this to update memory more accurately. Their approach, called AGMR, helps fix mistakes and improve memory use, leading to better task performance. Tests showed AGMR works better than older methods that rely only on texts.
self-evolving memoryretrieval-head attentionmemory utilizationmemory refinementinteractive decision-makingattention mechanismagent memorytask trajectoriescontext utilization matrixre-execution
Authors
Yechao Hong, Haiquan Qiu, Yaqing Wang, Quanming Yao
Abstract
Existing self-evolving memory systems mainly improve agent memory based on textual outputs, such as task trajectories and reflections. However, this text-based paradigm rarely incorporates internal mechanistic signals, leaving how retrieved memory is actually utilized during task execution underexplored. This limitation can lead to unreliable error attribution and hallucinated memory modifications. In this work, we show that retrieval-head attention provides a mechanistic signal for revealing segment-level memory utilization. By aggregating attention over memory segments and decision steps, we construct a context utilization matrix that exposes recurring memory-use patterns and indicates corresponding refinement strategies. Building on this observation, we propose Attention-Guided Memory Refinement (AGMR), a framework that uses utilization patterns revealed by attention to guide targeted segment-level memory updates. AGMR corrects or enhances memory for failed executions, simplifies memory for successful executions, and verifies each update through re-execution. Experiments on interactive decision-making benchmarks show that AGMR improves both task performance and memory efficiency over text-only memory refinement baselines. Code is available at https://anonymous.4open.science/r/AGMR_code-3262/