Federated networks improve clustering for high dimensional data
Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Data
Machine LearningArtificial Intelligence
Summary
Clustering groups similar data points together, which helps in organizing information without labels. This is hard when data is high-dimensional and spread out privately across different devices or locations. The authors extend a deep neural network approach to work in a federated setting, where data stays on user devices, by creating synthetic data and aligning the learned groupings better across devices. Their method works well even when the data on different devices is quite different. They tested their approach and found it improved clustering under these challenging conditions.
What this means in practice
- •For enterprise data teams: Cluster sensitive customer or operational data distributed across multiple locations without needing to pool the data centrally.
- •For medical software developers: Enable data analysis on decentralized and heterogeneous patient data while preserving privacy across healthcare institutions.
Authors
Morris Stallmann, Charalampos S. Kouzinopoulos, Marcin Pietrasik, Anna Wilbik
Abstract
Clustering high-dimensional data is a fundamental task in unsupervised machine learning with applications to a variety of domains. In the centralized data scenario, this task is commonly solved using deep clustering methods that utilize deep neural network architectures to learn clustering-friendly latent space representations. In Federated Learning, where data is distributed between clients and is private, deep clustering methods are less explored. In particular, recently introduced federated deep clustering methods, despite showing very promising performance, still fall short in reliably providing good performance if data across clients are non-identically-independently distributed. In this work, we introduce a generalization of Deep Clustering Networks to the federated scenario, named FedDCN, that simultaneously optimizes a reconstruction loss and a clustering loss. To ensure robustness and latent space alignment in non-identically-independently distributed data scenarios, FedDCN generates synthetic data augmentations, and its learning objective includes a geometric regularization for latent space alignment. Through experimental evaluation, the effectiveness of the approach under IID and non-IID assumptions is demonstrated, and future research directions are identified.