Memory selection method improves efficiency for large language model agents
Beyond Marginal Influence: Interaction-Aware Memory Selection for LLM Agents
Machine LearningArtificial Intelligence
Summary
Remembering all information for decisions can be inefficient for AI language models. The authors studied how to pick just enough memory so the model acts the same as with full memory. They made a test suite and a method called JointCore that finds such memory subsets efficiently. Their approach saves a lot of effort and memory without losing performance according to their tests.
What this means in practice
- •For ai system engineers: Efficiently reduce memory usage in language model agents while keeping decision-making behavior consistent by applying JointCore for memory selection.
- •For workflow automation developers: Streamline decision workflows by identifying smallest sufficient memory subsets that maintain full context for natural language tasks using the JointCore benchmark.
Authors
BHVSP Subrahmanyam, Neeraj Choudhary
Abstract
LLM-agent memory selection often uses retrieval rankings based on record relevance or individual influence. Such rankings cannot identify sufficient subsets under redundant or jointly necessary evidence. We study per-decision memory sufficiency: selecting a small subset that preserves full-memory behavior. To separate search error from model error, we introduce JointCoreBench, a benchmark with nine interaction families, six workflow domains, natural-language decisions, and exact sufficient-set oracles, and JointCore, a relevance cost-ordered group-deletion baseline that tests blocks before individual records. On 6,912 exact conditions, cost-ordered JointCore recovers a minimum-cost sufficient set in 100.0% of cases, reduces subset evaluations from 30.00 to 11.13 (62.9%), and achieves 87.8% active-memory savings. Across four 12B-70B model families and 432 tasks per model, its unweighted mean is 10.28 versus 16.00 model queries and 50.6% versus 43.1% minimum-cost recovery relative to cost aware singleton deletion. Both methods preserve full-memory behavior under the deterministic primary interface. Tests with physical deletion, UNKNOWN masks, PENDING sentinels, counterfactual values, and alternative labels identify interface dependent behavior. JointCore is an efficient benchmark baseline; the study does not establish deployment-ready memory compression.