Product monte carlo improves estimates from combined probability samples
Product Monte Carlo: Expectation Under a Product of Densities
Machine Learning
Summary
When trying to combine information from many separate sources to understand a bigger picture, it can be very hard to do the math correctly. The authors created a new method called Product Monte Carlo that uses samples from each source to estimate outcomes without needing to calculate complicated densities. They turned the problem into a type of regression, which also helps measure how confident we can be in the estimates. Their method works well in tests across many scenarios and scales better than earlier approaches.
What this means in practice
- •For machine learning engineers: Combine statistical models from multiple data holders efficiently without accessing their raw data by using Product Monte Carlo for federated learning tasks.
- •For computational biologists: Use Product Monte Carlo to compute expectations under complex probabilistic models from simulated biological data such as protein images without needing density functions.
Authors
Pietro Carlotti, Sifan Liu, Yuling Yao
Abstract
Aggregating independently computed sub-sample inferences is a well-known challenge in large-scale Bayesian computation, federated learning, and opinion pooling. The core problem is to approximate a distribution whose density is proportional to a product of individual densities. We propose Product Monte Carlo (PMC), an algorithm for computing arbitrary expectations under a product of densities using only Monte Carlo draws from individual densities. We turn the intractable product expectation into a regression problem by expressing it through a family of computable conditional expectations. Besides the point estimate, our fully Bayesian regression formulation provides coherent uncertainty quantification and principled diagnostics. We establish theoretical guarantees on consistency and characterize convergence rates for the PMC estimator. Across simulated and real-data experiments varying the number, dimensionality, and heterogeneity of the component distributions, PMC achieves higher estimation accuracy, better-calibrated uncertainty coverage, and improved scalability at a lower computational cost than existing methods. In addition, PMC requires no density evaluation, making it applicable to simulation-based inference, as we demonstrate in an application with protein images.