Deep generative models mainly mimic known crystal prototypes
Deep Generative Crystal Structure Prediction: A Benchmark Study and a Controlled Test of Prototype Dependence
Machine Learning
Summary
Predicting new crystal structures is important but challenging. The authors tested many deep learning methods designed to create crystal structures from scratch and compared them to simpler methods that copy from known templates. They found that the advanced models mostly reproduce what is already known, relying heavily on familiar structure families. Removing these familiar groups sharply reduced their accuracy, showing they don't yet create truly new structures reliably. The key challenge is improving these models’ ability to predict novel crystal structures beyond known examples.
What this means in practice
- •For materials scientists: Identify when generative models are likely to rediscover existing crystal structures rather than new ones, guiding experimental priorities.
- •For chemical engineers: Use benchmarked models to screen crystal prototypes efficiently during materials design, focusing on known structure families.
Authors
Lai Wei, Rongzhi Dong, Ying Feng, Madeline Miklos, Jianjun Hu
Abstract
Deep generative models are widely reported to enable de novo crystal structure prediction (CSP), but their capability has not been measured consistently against template-based methods. We evaluate 12 representative generative CSP models, spanning latent-variable, diffusion, flow-matching, autoregressive, and manifold random-walk architectures, against TCSP 2.0 on 180 test structures and a leakage-controlled subset of 46. All methods use identical structure-matching, symmetry, and consensus criteria. Template retrieval is the strongest single method, reaching 68.3% top-1 success; symmetry-aware EquiCSP (66.4%) and Uni-3DAR (62.9%) form the next tier. However, comparison with TCSP 2.0 shows that most structures correctly predicted by generative models are also correctly predicted by template substitution. Thus, the set of structures uniquely reachable by generation is small, limiting its practical advantage for discovering structures outside existing prototype libraries. To test the source of this performance, we removed entire stoichiometric prototype families from the training set and retrained the strongest generative model. Accuracy declined by 50-78% across four families, establishing that performance is substantially prototype-dependent. A small minority of structures survived removal of their prototype family, demonstrating a real but limited retrieval-independent predictive capacity. Present generative CSP models therefore function largely as implicit, softer-edged prototype libraries rather than genuinely de novo predictors. Enlarging this residual capacity, rather than aggregate match rate alone, is the central open problem.