Self improving agents adapt policy and memory recursively
MMLA-RSI: Recursive Self-Improvement through RTT and RTMA
Computation and LanguageMachine Learning
Summary
The paper studies how an intelligent agent can get better at solving a problem by changing both how it decides what to do and what experiences it keeps in memory. The authors propose two methods for self-improvement: one that updates the agent’s decision process during reasoning, and another that adapts the stored memory it trusts. These methods can improve themselves over time by creating successors that take over the updating role. The authors also provide a framework and examples showing that selectively keeping feedback can reduce errors, though some improvements remain to be tested more thoroughly.
What this means in practice
- •For autonomous system engineers: Design agents that improve their decision policies and memory stores during operation to handle hard tasks more effectively over time.
- •For software developers maintaining ai tools: Implement recursive update schemes that manage what data to keep or discard, improving long-term performance of AI models with feedback retention.
Authors
Junyi Zou, Avrova Donz
Abstract
An agent can improve its behavior within an unresolved problem by changing how it acts and which experience it retains. We propose MMLA-RSI, a recursive self-improvement scheme with two implementations in Memory-Mediated Learning Architecture: Reasoning-Time Training (RTT) changes the policy state $Φ$, while Reasoning-Time Memory Adaptation (RTMA) changes authoritative memory $M$. Each route also makes its adaptation method an object of improvement. A current numerical updater trains the policy proposing its successor; a current memory manager chooses the records available to the process proposing its successor. An independently admitted successor takes over the same role in the next generation. We give bounded executable rule languages, shared task/proposal interfaces, and one algorithm for either route or their composition. The RTT instance changes scalar schedules in a clipped-momentum update. The RTMA instance chooses NULL or a complete-row write with possible eviction. A fixed interpreter, trusted committer, and independent evaluator bound what can change. State versions, read closures, and conditional properties describe causal execution, accepted transitions, and finite work. Existing numerical-component studies motivate the construction. In four-candidate program selection, retaining LARC feedback updates reduces expected error by 24.65 and 36.65 percentage points for two trained initializations. In chronological continuous-integration replay, feedback binding is positive while retention increases loss. Completed SQL studies improve substantially over an unadapted substrate but do not establish the required additional online benefit. These results characterize their declared components. The two recursive routes and their joint realization are specified here; their beneficial recursive chains and discovery efficiency remain to be evaluated.