Bayesian federated learning estimates complex models while protecting privacy
A Bayesian Vertical Federated Learning Framework for Multivariate Reduced-Rank High-Dimensional Regression
Machine Learning
Summary
Combining data from different sources to build accurate models can be tough when privacy rules keep data separate. The authors present a new approach called BayesVFLReg that allows multiple sites to jointly analyze their data without sharing sensitive details. Their method compresses information into secure sketches and then runs a Bayesian model to find important relationships between many features and outcomes. This helps identify key variables even when data features are related to each other. The approach also includes a way to pick out relevant features efficiently and guarantees strong mathematical performance.
What this means in practice
- •For healthcare data teams: Use BayesVFLReg to jointly analyze patient data from multiple hospitals while ensuring each site’s data privacy and identifying key health indicators.
- •For financial fraud analysts: Apply the framework to integrate transaction features from different institutions without sharing raw data to detect fraudulent patterns across organizations.
Authors
Brigham Halverson, Sharmistha Guha, Jessica Bernard, Rajarshi Guhaniyogi
Abstract
Federated learning (FL) has emerged as a leading privacy-preserving framework for collaborative machine learning across decentralized environments. While considerable progress has been made in horizontal federated learning (HFL), where data with common features is distributed across sites, vertical federated learning (VFL), where sites share observations across distinct feature sets, remains less explored. Advancing Bayesian high-dimensional multivariate reduced-rank regression methods for VFL poses unique challenges: (a) stringent privacy regulations preventing local site data sharing, and (b) fitting local regressions overlooks essential modeling aspects like inter-variable correlations. In contrast HFL allows each site to fit a comparable model independently. We present a novel Bayesian VFL framework for multivariate high-dimensional reduced-rank regression, termed BayesVFLReg, which enables precise coefficient estimation while safeguarding both feature and response privacy. Participating sites use a shared random sketching matrix to compress local variables into privacy-preserving sketches. A central server collects these sketches where Bayesian multivariate reduced-rank regression uses Gaussian scale mixture priors. For feature selection, we introduce a single-step post-processing strategy based on mixture-model clustering of the absolute posterior coefficient means to distinguish signal from noise per response variable. BayesVFLReg is computationally scalable for large, high-dimensional datasets and facilitates efficient variable selection. Theoretically, we establish sharp non-asymptotic bounds on the posterior probability that the fitted density falls within a Hellinger ball centered at the true data-generating density. Comparative simulation studies and real-world data analyses show that BayesVFLReg reliably identifies sparse feature effects, even under feature correlation.