FairMean improves fairness and robustness in distributed learning
FairMean: Promoting Fairness in Distributed Learning under Label Poisoning Attacks
Machine LearningArtificial Intelligence
Summary
Distributed learning lets many users train a shared model by sending updates, but some users might send bad data on purpose to trick the system. To keep things fair for all users, the system tries to focus more on users with worse performance, but this can backfire if those users are attackers. The authors propose FairMean, a way to balance focusing on users with poor results while limiting the influence of attackers. Their tests show FairMean reduces differences in accuracy between users and improves performance for the worst-off users.
What this means in practice
- •For machine learning engineers: Improve model fairness and reliability by managing malicious client updates in distributed training.
- •For cybersecurity teams: Detect and mitigate label poisoning attacks in collaborative learning environments to protect model integrity.
Authors
Huigan Zheng, Jiaojiao Zhang, Yongxiang Liu
Abstract
Fairness-aware distributed learning prioritizes clients with large losses to reduce performance disparities, but label poisoning can create large losses, thereby inducing a fairness--robustness conflict. We propose FairMean to manage this conflict. FairMean weights client gradients using a bounded, nondecreasing function of local loss. The increasing weights prioritize high-loss clients to promote fairness, while the upper bound prevents excessive loss-induced amplification of poisoned-client gradients. In the absence of label poisoning, we show that minimizing the FairMean objective is more conducive to solution fairness than minimizing the standard average-loss objective. Under label poisoning, we establish an average-stationarity bound whose attack-dependent term is proportional to the square of the poisoned-client fraction. Experiments show that FairMean promotes fairness by reducing accuracy variance while improving worst-client accuracy.