Particle gflownets speed up training in complex discrete models
Particle GFlowNets: Rethinking Generative Marginalization Models
Machine Learning
Summary
Modeling complex systems that involve many choices often takes a long time to learn and simulate. The paper shows that two previous approaches, called MaMs and GFlowNets, are actually the same. Then, the authors improve the method by introducing a way to refresh the sampler at certain points using a statistical check. This helps the training process go faster, especially when working with big problems involving many combinations.
What this means in practice
- •For machine learning engineers: Accelerate training in models that generate complex discrete data by using particle GFlowNets for improved sampling efficiency.
- •For data scientists in combinatorial optimization: Improve inference speed when exploring large combinatorial spaces by applying particle GFlowNets to discrete stochastic models.
Authors
Tiago da Silva, Diego Mesquita, Salem Lahlou
Abstract
Generative Marginalization Models (MaMs) have been recently introduced as efficient neural sampling models for any-order autoregressive modelling of discrete distributions. By learning both the marginal and conditional probabilities of a persistent-block Gibbs sampler, MaMs enable fast posterior evaluation with a single neural network forward pass. While prior work has considered MaMs to be distinct from Generative Flow Networks (GFlowNets), a well-established paradigm for inference in discrete stochastic models, we show that they are equivalent. Then, we also extend MaMs' sampling strategy to non-autoregressive generative processes. In particular, we describe an automatic criterion for full-state rejuvenation of the Gibbs sampler, derived from the Gelman-Rubin statistic, which plays a key role in speeding up learning convergence. Our experiments show that our method, called Particle GFlowNets, markedly accelerates training in large combinatorial spaces.