Summary
The paper looks at how certain parts of a stock prediction model pay attention to relationships between different stocks. The researchers found that this attention behaves almost like it's spread evenly, but small special patterns within it are really important for making good predictions. They also show that simplifying attention to just a few key patterns works as well as the full version, but cutting out many connections randomly makes things worse. Interestingly, trying other graph-based methods did not help, and when scaling up to thousands of stocks, these cross-stock attention methods did not beat simpler models. This means the useful information in stock relationships is complex but can be captured efficiently with low-rank approaches.
multi-head attentioncross-sectional stock predictionNyström approximationlow-rank matrixspectral analysisgraph neural networksperplexityRank ICLSTMsparsification
Abstract
MASTER's inter-stock multi-head attention -- the module responsible for modeling cross-sectional stock relationships -- accounts for 42.5% of model parameters and 25% of predictive value. We systematically decompose this module and uncover a surprising structure: the learned attention is near-uniform (perplexity 278/300), yet forcing exact uniformity eliminates all cross-sectional discrimination. Spectral analysis resolves this paradox: the deviation from uniformity is low-rank (effective rank ~65, top-10 modes capture 96.5% of energy), explaining why sparse approximations consistently fail while Nystrom low-rank attention (m=32 landmarks) matches full O(N^2) attention at O(mN) cost -- certified equivalent via TOST at both N=300 (5 seeds, Rank IC p=0.003) and N=800 (10 seeds, Rank IC p=0.034). Additional findings include: (i) attention anti-correlates with return similarity (Spearman rho = -0.614; on the industry-labeled subset, -0.645 unconditionally and -0.627 after controlling for industry, beta, and volatility), suggesting complementarity-seeking rather than correlation mining; (ii) all graph-based alternatives degrade performance, with hard masking worse than complete module removal; and (iii) at N ~ 3,500 with adapted architectures, no cross-stock module (GCN, Nystrom, or MASTER-style pipeline) significantly outperforms a per-stock LSTM baseline (n=4 seeds), indicating that the benefits observed at smaller scales do not trivially transfer. These results establish that the inter-stock attention's value resides in a compressible, dynamic, near-global redistribution that rewards low-rank approximation but resists sparsification.