Financial world modeling improved through large scale market data analysis
Towards Financial World Modeling
Machine Learning
Summary
Understanding and predicting financial markets requires models that can capture many details like overall market trends, risks, and relationships between assets. The authors created a huge dataset with detailed stock market data spanning many years and used it to test different ways to build financial models. They showed that models that perform similarly in predictions can still have very different ways of representing market information. Their work lays groundwork for building better, comprehensive financial market models.
What this means in practice
- •For quantitative analysts: Use a massive dataset and evaluation methods to develop and compare market representation models for financial decision-making.$Commercial implications: Enables building better predictive financial tools for trading firms by providing comprehensive training data and evaluation protocols.
- •For economic forecasters: Incorporate broad market state representations to improve predictions of economic indicators under varying financial conditions.
Authors
Humzah Merchant, Alec Guthrie, Simon Mahns, Randall Balestriero, Bradford Levy
Abstract
Building a world model requires a state representation useful for planning and decision-making---potentially over tasks unknown at training time. In the context of financial markets, planning and decision-making may require a model to reason about market-wide conditions, asset-specific expected returns, liquidity, volatility, and cross-asset relationships. Yet financial representation learning has largely been evaluated on individual predictive tasks, oftentimes on a single time period using comparatively narrow datasets. We address this through three primary contributions. First, we introduce Market-1T, a dataset containing nearly one trillion observations across U.S. equities from 2008 to 2025 at 1 Hz resolution. Second, we develop and implement a rigorous evaluation protocol. Third, we conduct a systematic large-scale study of financial representation learning, comparing 18 encoder-training strategies across nearly two decades of market regimes. We evaluate learned representations both by their predictive utility on common finance tasks and through probes of latent structure. We find that encoders with similar predictive performance can organize market state very differently. Collectively, we establish a foundation for training and evaluating financial market representations in support of world models such as DINO-WM, V-JEPA 2, and LeWM.