State-space model reveals market dynamics inside order book data
Understanding the Inner Workings of a State-Space Foundation Model for Limit Order Books
Machine Learning
Summary
Financial markets use records called limit order books (LOBs) to track buy and sell orders. This paper shows that a particular AI model can learn how these orders influence price changes by analyzing message sequences. The authors found the model builds a meaningful internal understanding that applies to many stocks, not just those it was trained on. They also demonstrated that nudging this model's internal state changes its predictions in ways that match known market behaviors.
What this means in practice
- •For trading system developers: Improve market simulation components by incorporating models that internally represent order-flow and price dynamics for more realistic prediction of market messages.
- •For financial risk analysts: Use internal representations of market dynamics from models to monitor indicators like order-flow imbalance and volatility for better short-term risk assessment.
Authors
Valentin Mohl, Alexandre Bismuth, Aramis Marti-Shahandeh, George Nigmatulin, Kang Li, Stefan Zohren, Jakob Foerster, Mihai Cucuringu, Anisoara Calinescu
Abstract
Autoregressive foundation models offer state-of-the-art capabilities to simulate the market by generating the message flow of limit order books (LOBs). Yet, without understanding their inner workings, research directions are hard to narrow and practical use in real-world trading environments remains uncertain. This study therefore asks what these models actually learn about market microstructure. We find that, similarly to games where sequence models build an internal representation of the board, an 8.2M-parameter Mamba-3 state-space model trained autoregressively on the tokenized message stream of eight large-cap US equities learns an internal representation of the market and its dynamics. In particular, order-flow imbalance, a central driver of short-horizon price moves, is decodable at $R^2 \approx 0.59$, far above controls and baselines. This representation extends to unseen stocks, suggesting that these models learn general representations rather than ticker-specific patterns. The model also generalizes across metrics beyond order-flow imbalance: results hold across traded volume and volatility. Additionally, the representation is causally steerable: pushing the state along a feature's axis shifts the model's next-message beliefs as that feature implies, monotonically in dose, consistent with the emergence of a world model.