MemTrial improves portfolio decisions by judging memory reliability
MemTrial: Learning When to Trust Memory in LLM Portfolio Agents
Artificial Intelligence
Summary
Managing money with AI models involves learning from past experiences, but in finance, most results just reflect overall market moves, not what each experience actually changed. The authors present MemTrial, a system that tests the impact of each past experience by comparing decisions made with and without it, filtering out market-wide effects. MemTrial combines these tests using a method that values each experience's unique contribution and pools data over time to make safer, more reliable investment decisions. This approach improves performance on multiple benchmarks and avoids big losses when past experiences aren't very helpful.
What this means in practice
- •For quantitative finance teams: Enhance algorithmic trading by distinguishing useful past experiences from general market trends to improve portfolio decision making.
- •For investment data scientists: Develop more robust portfolio agents that minimize risks by cautiously integrating memory-based insights validated against unseen data.
Authors
Guanghao Wu, Zhuo Cai, Shoujin Wang
Abstract
Large language model (LLM) agents for portfolio management learn from experience: they credit each experience in their memory with the outcome of the decisions that used it. In financial markets, however, this outcome mostly reflects the market move shared by all decisions on that date, so the credit tracks the market rather than the experience, and these agents often do worse than simply holding the equal-weight (1/$N$) portfolio. We ask how an agent can credit an experience with what it changes, and answer it by putting memory on trial: drafts of the same decision with and without an experience face the same market, so the outcome they share cancels in their difference. Our agent, MemTrial, drafts each decision with eight combinations of its retrieved experiences, chosen by a fractional factorial design, and credits each experience with its Banzhaf value, the average of these differences. As each date occurs once and each draft is a noisy LLM sample, these credits are noisy and may not hold on new dates. MemTrial therefore pools them across dates and similar experiences with a hierarchical Bayesian model, acts on them only after they have predicted unseen dates, and otherwise stays anchored at a conservative reference such as 1/$N$. On four benchmarks, MemTrial not only benefits from experiences that matter (the best of 15 methods on a semi-synthetic benchmark with known experience quality) but also limits its losses when its values do not hold (at most 2.2\% below 1/$N$ on PortBench and InvestorBench, against 15--38\% for the best experience-learning agent). Averaged over five settings, it improves the utility of the best experience-learning agent by 21.2\%, and with eight LLMs it beats every LLM-based baseline on InvestorBench.