Fixed-point neural sampler reduces mode collapse in discrete data
Fixed-point neural samplers on discrete spaces
Machine Learning
Summary
Sampling from complex sets of discrete options without actual data is difficult. The authors propose a new neural network method called Discrete Gibbs Iterative Neural Sampler that improves sampling by reducing common problems like mode collapse, where the model ignores some possibilities. Their approach also scales well to high-dimensional problems and can handle different sampling tasks efficiently. This method helps in estimating complex systems like alloy phase diagrams more accurately.
What this means in practice
- •For materials engineers: Generate reliable samples of material phase configurations to aid the design of new alloys based on estimated phase diagrams.
- •For machine learning engineers: Create stable and scalable discrete data generative models that avoid mode collapse for improved model training and data generation.
Authors
Jiajun He, Denis Blessing, Mouyang Cheng, Yuanqi Du, Carles Domingo-Enrich
Abstract
Sampling from discrete, unnormalized distributions without access to data is a challenging problem. Neural samplers offer a promising approach by training generative models from density evaluations directly. Despite recent progress, existing discrete neural samplers are prone to mode collapse, come without convergence guarantees when trained via fixed-point iterations, and are often tied to a specific reference process such as masked or uniform diffusion. In this work, we introduce Discrete Gibbs Iterative Neural Sampler, a fixed-point neural sampler that addresses these limitations, enabling efficient, scalable learning, substantially reducing mode collapse in practice. Our framework builds on masked diffusion and also extends to transport between pairs of distributions. We demonstrate that the resulting method scales effectively to high-dimensional systems, supports amortized sampling across different conditions, and enables accurate estimation of alloy phase diagrams.