Distributed teams optimize complex tasks by sharing compact summaries
GUIDE-FBO: Guidance via Uncertainty Intervention and Distributional Exchange for Federated Bayesian Optimization
Machine Learning
Summary
Optimizing costly and unknown processes often requires multiple teams working together without sharing sensitive data. The authors present a method where teams exchange small summaries describing their best guesses about the solution instead of raw data. These summaries help each team improve its own decisions while respecting communication limits and differences in tasks. Their approach performs well on both simulated and real-world problems, maintaining efficiency despite varying team needs.
What this means in practice
- •For machine learning teams: Coordinate multiple teams to tune expensive models collaboratively without sharing raw training data by exchanging compact statistical summaries.
- •For industrial process engineers: Optimize distributed manufacturing or chemical processes with limited communication by sharing summaries of best operating parameters rather than raw results.
Authors
Jintao Wei, Chenxi Li, Songhao Wang
Abstract
Federated Bayesian Optimization (FBO) enables distributed agents to collaboratively optimize expensive black-box objectives without sharing raw local observations. However, effective knowledge transfer remains challenging under communication constraints and task heterogeneity. We propose GUIDE-FBO, in which agents exchange compact distributions over the locations of their respective optima inferred from local Gaussian process (GP) posteriors, rather than raw observations, query points, or surrogate parameters. The server merges and reweights these distributional components before returning a subset to each agent. Each agent then constructs a Federated Interventional GP (FI-GP), which preserves the local posterior mean and spatially rescales its covariance for local decision making. For the upper confidence bound (UCB) instantiation, GUIDE-UCB, we prove that any bounded FI-GP uncertainty intervention preserves the leading-order cumulative regret rate of standard GP-UCB. When the transferred distributions place greater support near an optimum than in a suboptimal region, selecting the latter requires greater local posterior uncertainty. Experiments on 12 synthetic benchmarks and three real-world optimization tasks show that GUIDE-FBO remains effective across settings ranging from homogeneous to severely heterogeneous. Ablation results highlight the importance of spatially localized uncertainty intervention, while the communication analysis shows that GUIDE-FBO exchanges only compact distributional messages.