FedBST improves unsupervised federated image classification quality
FedBST: Bilateral Self-Training for Unsupervised Federated Classification
Computer Vision and Pattern Recognition
Summary
Classifying images without labels across different devices is hard when the data varies a lot. The paper shows that existing methods miss useful information from uncertain or confusing images. The authors propose a new approach called FedBST that uses both confident and ambiguous images to better learn patterns. They also combine visual and text information during prediction to adapt to each device’s data. Their method improves the accuracy of federated classification compared to earlier techniques.
What this means in practice
- •For mobile app developers: Improve image classification accuracy on user devices without needing labeled data by handling diverse and ambiguous images.
- •For enterprise data teams: Deploy federated learning systems for image classification that better adapt to varying data distributions across multiple sites.
Authors
Aijing Yu, Kuangpu Guo, Jian Liang, Xiao-Yu Zhang
Abstract
Unsupervised federated learning with Vision-Language Models (e.g., CLIP) offers a promising solution for label-free classification, yet its performance often degrades under severe non-IID shifts. A pioneering method, FedCoPL, typically relies on self-training strategies that exclusively prioritize high-confidence predictions for pseudo-labeling. However, this selective approach neglects the information of ambiguous samples (often referred to as ``waist" samples), leading to the underutilization of data and biased decision boundaries. To address this limitation, we propose FedBST, a bilateral self-training framework that effectively exploits both high-confidence and ambiguous data. Specifically, for local optimization, we introduce uncertainty-aware confidence penalization (UACP). UACP targets ambiguous samples by penalizing overconfident predictions to implicitly promote higher uncertainty, thereby preventing erroneous pseudo-labeling. At inference, we employ a visual-text ensemble strategy. This approach aggregates uncertainty-aware local visual prototypes and ensembles them with textual prototypes, achieving lightweight adaptation to local distributions via prototype-based calibration. Experiments demonstrate that FedBST achieves consistent improvements over existing methods with standard federated aggregation protocols.