AlphaRJM improves formulaic alpha discovery using reward-jump memory

AlphaRJM: Reward-Jump Memory for Stochastic Return-Guided Alpha Discovery

Machine Learning

Summary

Finding formulas that predict stock returns is tricky because feedback only comes after a complete formula is evaluated, making it hard to judge partial progress. The authors introduce AlphaRJM, which keeps track of rewards from complete formulas in a special memory that only updates when a formula finishes. It also models future payoffs with a technique that accounts for uncertainty. Together, these approaches help select better actions when building formulas and lead to more stable and better predictions in stock markets.

What this means in practice

  • For quantitative finance teams: Improve stock selection models by using AlphaRJM to better evaluate and select predictive formulas with delayed reward feedback.
  • For algorithmic trading developers: Integrate AlphaRJM’s stochastic return modeling to enhance action selection under uncertainty in automated trading strategy construction.

Authors

Sayan Dhan, Selvaraju Natarajan

Abstract

Formulaic alpha discovery is a pool-dependent symbolic search problem in which informative feedback is observed primarily when a complete expression is evaluated. This delayed feedback creates two coupled difficulties: the retained alpha pool does not preserve the full history of realized evaluation feedback, and the value of an intermediate construction action is uncertain because its consequence depends on the formula eventually completed. We introduce AlphaRJM, which addresses these difficulties through Reward-Jump Memory, an event-driven latent state that remains fixed during token construction and updates only at terminal evaluation events using the realized pool reward and evaluation outcome, and an action-conditioned SDE return critic that represents future discounted discovery returns with stochastic particles. The particles guide action selection through their mean and uncertainty and are learned using a distributional Bellman objective combining energy-distance matching, mean calibration, and jump regularization. Empirically, AlphaRJM delivers strong and stable gains across multiple equity universes, forecasting horizons, and random seeds, while ablations confirm the complementary roles of persistent evaluation history, stochastic return modeling, and distributional supervision.