Hippo helps ai agents improve without changing their code
Hippo: Training-Free Agent Self-Evolution via Hippocampal Replay
Artificial Intelligence
Summary
Artificial intelligence agents often struggle to learn from their past attempts because it's hard to tell if a success was due to good planning or just luck. This paper introduces Hippo, a memory system inspired by how the brain's hippocampus works, which helps agents compare different tries to remember useful decisions and fix mistakes. Hippo stores and updates these memories without needing to change the AI itself, leading to better performance on various tasks with fewer memories kept. The authors show that even very capable AI models can benefit from this kind of memory-based learning.
What this means in practice
- •For ai application developers: Enhance AI agents by enabling memory-based reasoning improvements without retraining or modifying the underlying models.
- •For software developers: Implement AI assistants that learn from past interactions through selective memory updating, improving task success over time without costly model updates.
Authors
Yutong Song, Jiang Wu, Yuxuan Ren, Tony Qin, Baoyuan Wang
Abstract
Motivated by recursive self-improvement (RSI), we study how LLM agents can learn through external memory without updating model parameters. Extracting useful experience from a single attempt is difficult: flawed reasoning can produce success, while poor execution can obscure a sound strategy. Preserving its value over time is also difficult: our accumulation baseline without consolidation initially improves but eventually performs below a no-memory baseline. Our key idea is to compare attempts at the same task to identify reusable decisions and refine when they apply. We introduce Hippo, a memory framework inspired by selective hippocampal encoding and replay that learns from these comparisons when shortcomings are diagnosed. It compares the resulting experience with existing memory to merge overlaps and resolve conflicts through explicit applicability conditions. This allows prior guidance to be revised without storing every lesson as a separate entry. Experiments on SWE-bench Verified, WebArena, and Mind2Web show performance gains in all three settings and substantially fewer retained experiences than the accumulation baseline. The gains extend across the evaluated base models of varying capabilities, indicating that memory-based adaptation remains useful even for highly capable models.