Distributed learning method improves privacy and accuracy on tabular data
Geodesic-SVDD Latent Mixup for One-Shot Distributed Tabular Learning
Machine Learning
Summary
Learning from data across different places without sharing sensitive details is important for fields like healthcare and finance. The authors created a new method called Geodesic-SVDD that helps computers learn patterns on structured data by representing client data as shapes on a sphere. This approach prevents problems common in existing techniques and allows one-time sharing of models to protect privacy. Their method showed better performance than other federated learning methods on several tests, especially when data labels are unevenly distributed.
What this means in practice
- •For financial fraud analysts: Build privacy-preserving fraud detection models that learn from multiple institutions' tabular data without sharing raw data.
- •For healthcare data teams: Deploy one-shot federated learning for patient diagnostics from distributed health records while handling uneven label distributions.
Authors
Andrea Moleri, Christian Internò, Markus Olhofer, Barbara Hammer, Ali Raza
Abstract
Distributed Learning on structured data is a key and widely adopted approach in privacy-sensitive sectors, ranging from financial fraud prevention to federated healthcare diagnostics. While Gradient Boosted Decision Trees are widely used in centralized settings due to their efficiency, they are incompatible with standard gradient-based federated learning protocols because of their non-differentiability. Conversely, standard deep classifiers that are amenable to federated optimization often suffer from catastrophic forgetting under label distribution skew. In this work, to address these gaps we propose Geodesic-SVDD, a framework for Distributed Spherical Prototype Learning that recasts each client's task as one-class density estimation on the hypersphere, so that local models can be aggregated as probabilistic manifolds in a single round. Specifically, for each client, we employ a Residual Spherical Encoder and Geodesic Latent Mixup that enforces strict hyperspherical constraints ($\|φ(x)\| \equiv 1$), preventing the manifold collapse observed in linear approximations. Furthermore, leveraging this insight, we propose a novel One-Shot "Product of Experts" protocol that enables privacy-preserving aggregation without iterative synchronization. Finally, we mechanistically analyze the trade-off between interpolation strategies, demonstrating that while linear mixup offers an enticing computational shortcut, it destabilizes optimization gradients by orders of magnitude ($10^9\times$). Across five tabular benchmarks and four levels of label skew, Geodesic-SVDD attains the highest mean AUROC among seven federated methods, outperforms federated tree ensembles by up to 21 AUROC points on Covertype, and its One-Shot variant matches centralized Deep SVDD under IID and mild skew with a single communication round.