A unifying view of contrastive learning, importance sampling, and bridge sampling for energy-based models
2026-04-09 • Computational Engineering, Finance, and Science
Computational Engineering, Finance, and Science
AI summaryⓘ
The authors study energy-based models (EBMs), which are probabilistic models difficult to work with because parts of them can't be calculated directly. They show how several existing methods used to estimate parameters in EBMs, like noise contrastive estimation and bridge sampling, are actually connected and sometimes equivalent. This new understanding helps explain why some methods work well and points to ways to make them better. The authors also provide their MATLAB code to help others verify their findings.
energy-based modelsnoise contrastive estimationreverse logistic regressionmultiple importance samplingbridge samplingparameter estimationlikelihood intractabilityprobabilistic modelsstatistical efficiency
Authors
Luca Martino
Abstract
In the last decades, energy-based models (EBMs) have become an important class of probabilistic models in which a component of the likelihood is intractable and therefore cannot be evaluated explicitly. Consequently, parameter estimation in EBMs is challenging for conventional inference methods. In this work, we provide a unified framework that connects noise contrastive estimation (NCE), reverse logistic regression (RLR), multiple importance sampling (MIS), and bridge sampling within the context of EBMs. We further show that these methods are equivalent under specific conditions. This unified perspective clarifies relationships among existing methods and enables the development of new estimators, with the potential to improve statistical and computational efficiency. Furthermore, this study helps elucidate the success of NCE in terms of its flexibility and robustness, while also identifying scenarios in which its performance can be further improved. Hence, rather than being a purely descriptive review, this work offers a unifying perspective and additional methodological contributions. The MATLAB code used in the numerical experiments is also made freely available to support the reproducibility of the results.