AlphaPareto improves trading signal discovery with multi-objective learning
AlphaPareto: Formulaic Alpha Discovery with LLM-Guided Multi-Objective Reinforcement Learning
Artificial IntelligenceMachine Learning
Summary
Finding good trading signals that work well together is very hard. The authors present AlphaPareto, a method that helps computers create and improve these signals by looking at several qualities at once, like how well they predict, how stable they are over time, and how different they are from each other. They also use a large language model to better understand the current collection of signals, so the process adapts as things change. Tests with real data show AlphaPareto works better than previous methods.
What this means in practice
- •For quantitative trading teams: Generate trading signal formulas that balance prediction accuracy, stability, and diversity to improve trading strategies.
- •For financial software developers: Integrate a multi-objective reinforcement learning approach to enhance tools that design and select financial alphas dynamically.
Authors
Yingbo Zhao, Zeyu Yang, Zhoufan Zhu
Abstract
Formulaic alpha discovery is a core challenge in quantitative trading, as identifying alphas that work well together remains difficult. Recent reinforcement learning (RL) methods formulate this task as a Markov decision process (MDP), but two important issues remain unresolved. First, as the alpha pool evolves, the reward function changes accordingly, making the MDP inherently non-stationary. Second, most existing methods optimize a single objective, typically predictive power, while ignoring other important properties of a high-quality alpha pool. Motivated by these challenges, we propose AlphaPareto, an RL method for formulaic alpha discovery. To address non-stationarity, AlphaPareto augments the state to include both the alpha under construction and the current alpha pool, and applies a large language model (LLM) to encode the pool. This design allows the agent to adapt to the evolving search environment. To overcome the limitation of single-objective reward design, AlphaPareto replaces the scalar reward with a multi-objective vector-valued reward that simultaneously captures predictive power, temporal stability, perturbation robustness, and diversity, and optimizes these objectives through a Pareto-regularized learning procedure. Empirical applications to real-world datasets show that our AlphaPareto method outperforms its competitors.