Thompson sampling improved by diffusion transformers with flexible priors

Thompson Sampling using Prior-fitted Diffusion Transformers

Machine LearningArtificial Intelligence

Summary

Thompson sampling is a method used to find the best option when you don't know much about what you are testing. Traditionally, it worked mainly with simple assumptions called Gaussian processes, which limited how well it could model complex situations. The authors introduced a new way using diffusion transformers—advanced AI models—trained on more flexible assumptions that can describe a wider range of functions. Their approach allows better guessing about unknown functions and improves finding the best solutions in tests compared to traditional methods.

What this means in practice

  • For machine learning engineers: Use the proposed diffusion transformer models to optimize complex systems where standard Gaussian assumptions fail, enhancing black-box optimization performance.
  • For industrial optimization teams: Improve global optimization of engineering and design processes by applying non-Gaussian prior models that better match realistic function properties.

Authors

Sihwa Park, Jingsen Zhu, Vinamr Jain, Sheng-Yen Chou, Alexander Terenin

Abstract

Thompson sampling is natural randomized strategy for optimizing unknown functions in a black-box manner. In continuous global optimization, Thompson sampling has largely been restricted to Gaussian process priors, as these are essentially the only class for which computing the posterior distribution has been tractable in practice. In this work, we introduce \emph{prior-fitted diffusion transformers}, which are diffusion models that perform Thompson sampling in-context at inference time, after being pre-trained using the prior-fitted network paradigm. To apply these models, we develop a set of practical non-Gaussian priors for general black-box functions---including those that have characteristics that cannot be achieved under Gaussianity, such as priors over unimodal functions. We also quantify the tradeoff between model size and the number of pre-training iterations, and show that these models follow scaling laws akin to those observed in transformers defined over other modalities. Across a set of global optimization benchmarks, we show that prior-fitted diffusion transformers with non-Gaussian priors can achieve improved performance compared to traditional pipelines built on Gaussian processes.