MemRiskBench evaluates long-term risks in memory use by AI agents
MemRiskBench: Trace-Aware Risk-Preserving Evaluation for Long-Horizon LLM Agents
Artificial Intelligence
Summary
Large language model agents keep memories across sessions, but this can cause problems like outdated facts or accidental data leaks. The authors created MemRiskBench, a test that tracks these risks clearly, unlike usual tests that hide rare but serious mistakes. Their test uses detailed logs and rules to catch different risk types without relying on the AI itself to judge. They also made a way to pick a small, smart set of tests that still finds all major problems, making risk evaluation faster and cheaper. This helps developers better understand and improve the safety of AI agents that remember past interactions.
What this means in practice
- •For ai safety teams: Detect and monitor specific memory-related risks in deployed LLM agents using a transparent, automated benchmark with detailed trace analysis.
- •For software engineers: Use a reduced, risk-preserving test set to efficiently evaluate memory safety in long-term LLM agent development without needing costly human review.
Authors
Jianhua Jiang, Dongbo Yuan, Weihua Li
Abstract
Long-horizon LLM agents accumulate memory across sessions, creating sparse but high-impact risks: stale facts, conflicting updates, cross-user leakage, revoked-memory reuse, and constraint decay. Standard aggregate scores hide per-risk failure rates--a model achieving 78% average accuracy may still leak data in 4% of episodes--and benchmark compression preferentially discards the rare high-severity events that distinguish a mostly-working model from one that occasionally causes harm. We present MemRiskBench. The primary contribution is a five-category risk taxonomy (plus one documented, unscored category) operationalized by deterministic trace grounded checks, instantiated as a 120-episode scripted benchmark with full trace logging and no LLM-as-judge on the pass/fail path, evaluated on five locally run quantized instruction-tuned models. Second, a risk-preserving subset selector: a coverage-constrained greedy selector on deterministic trace-derived features that retains full ranking (Spearman rho = 0.975, deterministic; CI collapses to a point estimate with zero bootstrap variance), risk coverage (1.0), and high-risk model detection (1.0) at a 20% subset size, reducing compute 5x. Unlike ranking-only subset selectors, this selector additionally preserves risk-type coverage and high-risk detection using trace-grounded deterministic features that do not require an LLM judge. All episodes, traces, the scoring implementation, and the selector are released to support reproducible evaluation and risk assessment of deployed LLM agents