Clustered federated learning with offline client grouping cuts communication cost
RIPPLE in Still Water: Zero-Shot Clustering in Federated Learning with Wavelet Scattering Transform
Machine Learning
Summary
Federated learning lets many devices train a shared model without sending raw data, but it struggles when the data is very different across devices. This paper introduces RIPPLE, a way to group devices into clusters based on their local data before training begins, using a technique called the Wavelet Scattering Transform. By doing this grouping offline, RIPPLE reduces communication steps and keeps client data safer. The authors show that RIPPLE trains specialized models for each group efficiently and outperforms other methods in several test scenarios.
What this means in practice
- •For mobile app developers: Personalize models on client devices without extra communication by assigning new clients to learned clusters using offline data summaries.
- •For edge device integrators: Deploy federated models tailored to client data groups that reduce communication overhead and improve accuracy in heterogeneous environments.
Authors
Alessandro Licciardi
Abstract
Clustered Federated Learning (FL) partitions a client population into groups of similar local distributions and trains one specialized model per cluster, mitigating client drift that degrades single-model methods under non-IID data. Prior methods discover cluster structure inside the training loop through gradient similarity, loss evaluation, or EM-style updates, thus increasing communication overhead, exposing gradients to inversion attacks, and providing no mechanism to assign clients absent from training. We propose RIPPLE, a clustered FL framework in which cluster assignment is computed entirely offline from a spectral characterization of each client's local data: a variance-weighted principal-component prototype embedded via the Wavelet Scattering Transform and decoded by a Gaussian Mixture VAE trained server-side on synthetic client populations before federation begins. Per-round communication cost matches FedAvg exactly, and a client absent from training obtains a personalized model from a single forward pass, without gradient computation, model evaluation, or extra communication round. We prove that the gap between RIPPLE's surrogate clustered objective and the oracle is bounded by a computable quantity decaying with client sample size and independent of federation duration; per-cluster convergence matches the minimax-optimal rate for non-convex smooth objectives. Across five benchmarks spanning controlled and realistic heterogeneity, RIPPLE consistently outperforms all baselines, with margins growing on the most realistic partitions.