Scalable sampling method improves inference under faulty simulators
Scalable Diffusion SBI for Compositional Inference under Simulator Misspecification
Machine Learning
Summary
Simulations are sometimes used to understand complex systems, but making accurate guesses from many different observations can be very hard, especially when the simulation is slightly wrong. This paper presents a new way to combine data from different situations efficiently without retraining the model each time. It also introduces a method to adjust the model when it doesn’t match the real data well, improving predictions. The approach was tested on various models and on real biological data, showing better accuracy and flexible use.
What this means in practice
- •For computational biology teams: Improve accuracy of mechanistic models by fine-tuning inference with variable experimental data under model mismatch.
- •For machine learning engineers: Use a single pretrained diffusion model for flexible inference with different data groupings without retraining the model.
Authors
Vincent D. Zaballa, Elliot E. Hui
Abstract
Simulation-based inference is challenging when many heterogeneous observations must be composed, hierarchical latent structure must be preserved, and the simulator is misspecified relative to observed data. We develop sampling and fine-tuning methods for diffusion-based inference in design-conditional settings, where the same simulator is queried across different experimental conditions $ξ$. We extend compositional score-based inference with a continuous-time diffusion coefficient that accounts for the number of observations, avoiding Jacobian and auxiliary-covariance corrections. We introduce Hierarchical Blockwise Diffusion Sampling (HBDS), which infers shared parameters and group-specific latent states using a single pretrained model, with the hierarchy specified only at sampling time. Together, these methods support variable observation sets and groupings without retraining. To address misspecification, we introduce path-regularized fine-tuning that adapts the learned likelihood to observations and transfers corrections to posterior inference. Using Girsanov's theorem, we quantify path divergence between pretrained and fine-tuned models across experimental designs and interpret it alongside predictive errors to distinguish candidate misspecification correction from unnecessary adaptation. We evaluate compositional sampling on exact-score Gaussian and Simple Likelihood, Complex Posterior benchmarks, HBDS with analytic and learned scores on a controlled hierarchical model, and fine-tuning and localization on a separate analytic model with known design-dependent discrepancy. Finally, we apply the framework to 940 measurements across four cell lines in a mechanistic Bone Morphogenetic Protein signaling model, where fine-tuning improves posterior-predictive accuracy relative to the pretrained model and shifts posterior marginals toward the least-squares reference while retaining spread.