Cross-silo federated learning expands models with little new data
When the Cross-Silo Federation Goes Offline: Continual Learning for Site Onboarding with Limited Unlabeled Data
Machine LearningArtificial IntelligenceDistributed, Parallel, and Cluster Computing
Summary
Sometimes an organization wants to improve a machine learning model but doesn’t have enough labeled data, and other organizations with more data can’t share it. The authors study a way where a new site can join a group after the original training finished, with its unlabeled data, and have the model learn new conditions without losing knowledge. They use a three-step method that spots new types of data, groups them, and connects them to known classes using shared features. Their method balances accuracy on old and new tasks better than other approaches even with very few labeled examples per new category.
What this means in practice
- •For industrial data teams: Enable new factory sites to integrate unlabeled sensor data into existing predictive maintenance models without sharing raw data or losing past knowledge.
- •For medical device teams: Add new clinical sites’ unlabeled patient data to hospital-wide diagnostic models while preserving previously learned disease patterns.
Tested on one dataset.
Authors
Ahmadreza Eslaminia, Klara Nahrstedt, Chenhui Shao
Abstract
An organization often holds too little labeled data to train a model that generalizes, and the records that would supply the rest sit with organizations that cannot release them. Cross-silo federated learning offers a way through, since participants exchange model parameters rather than records, but it ordinarily settles two aspects of the arrangement in advance, the participating sites and the classes the model can predict, and deployment can breach both. A new site joins after training, once the established sites have finished their engagement and gone offline, and its records arrive unlabeled, mixing conditions the model already recognizes with conditions no participant has observed. We present an autonomous three-stage procedure that expands the model entirely at the joining site: reconstruction experts screen for novelty, clustering separates the flagged records into candidate conditions, and class means describe the old classes, all inside one shared representation. Those classes were learned from records that never leave their owners, so the usual defenses against forgetting are unavailable, and the procedure supplies the evidence they would have carried from either of two dissimilar sources, prototypes held by the federation or records held by the joining site. On a real industrial condition-monitoring dataset, run end to end with no label consulted, either source holds old-class accuracy at 0.868 or above with forgetting at most 0.063, and the two differ by 0.021, so a configuration can be chosen by the disclosure it permits rather than the accuracy it delivers. Both keep old- and new-class accuracy in balance where every alternative we measure gives up one for the other, and both retain more of the old classes than distillation- and regularization-based baselines. The balance still holds with only 6 labeled records per arriving condition and 3 retained per old class.