Financial research agents tested with a new reliable promotion protocol
Can We Stop the Backtest Yet? Anytime-Valid Promotion Tests for Adaptive Financial Research Agents
Artificial Intelligence
Summary
Financial research agents can quickly test and update trading rules, but this speed risks false positives when deciding which rules are good. The authors introduce PROMOTE, a method that limits the chance of mistakenly promoting weak or bad rules by carefully tracking testing resources and outcomes over time. This approach works even when many rules appear adaptively and are monitored indefinitely. They tested PROMOTE with historical financial data and different null models, showing it greatly reduces false promotions compared to traditional methods while still detecting real improvements.
What this means in practice
- •For quantitative trading teams: Increase confidence in automated selection of trading rules by limiting false promotions during adaptive backtests.
- •For financial data scientists: Analyze large sets of adaptive trading hypotheses with controlled error rates when monitoring rules over time.
Authors
Eric Benhamou, Jean-Jacques Ohana, Beatrice Guez, Ethan Setrouk, Chamyl Saadi, Thomas Jacquot
Abstract
Financial research agents can generate and monitor trading rules faster than a human can document the resulting selection process. Candidates appear at data-dependent times, weak rules are abandoned, and promising ones are watched until they cross a reporting threshold. We introduce PROMOTE, a protocol that assigns every candidate predictable testing wealth at birth and accumulates a cost-aware e-process only from subsequent returns. If lifetime allocations sum to at most one, a weighted Ville argument bounds the probability of ever promoting a rule whose declared bounded net score has always had nonpositive conditional mean. The result permits dependent candidates, adaptive births, and indefinite monitoring. We audit the implementation in two settings. A retrospective pseudo-out-of-sample replay contains 24 panel--rule hypotheses, 3,000 shared-sign null paths, injected alternatives, and public ETF and industry data. Repeated unadjusted tests produce a family crossing in 94.5% of null paths, versus 0.0% for PROMOTE; at a 15-bp aligned ETF-component perturbation, PROMOTE has 70.1% target power and median crossing session 2,327. A separate exact mean-zero experiment lets an agent select candidate identities and birth times from past scores; the funded gate has 1.77% false-family crossings across 3,000 programs. The market replay produces zero promotions after costs. The certificate controls historical false promotion under the declared null; it is not a certificate that an earlier edge remains profitable at deployment. AI-use disclosure. Generative AI tools, including OpenAI Codex, assisted manuscript preparation. For this preprint version, Codex assisted with LaTeX editing, author and venue metadata, compilation, and layout checks.