FedHV reduces communication for multi-objective federated learning
FedHV: Low-Overhead Hypervolume Weighting for Federated Multi-Objective Optimization
Machine Learning
Summary
Federated learning involves training a shared model across many devices while keeping data private. When optimizing for multiple goals, balancing these competing tasks can be tricky and often requires a lot of communication between devices and the server. The authors introduce FedHV, a new method that efficiently assigns importance weights to tasks using just a few extra numbers sent from devices, cutting down communication costs. Their method performs well across various image recognition tests, improving accuracy and fairness among tasks.
What this means in practice
- •For mobile app developers: Train models on user devices for multiple prediction tasks with less communication and better task balance.
- •For edge computing teams: Deploy multi-objective learning on resource-limited edge devices while reducing communication overhead and maintaining accuracy.
Authors
Amirardalan Dehghanpour, Seyed Mohammad Azimi-Abarghouyi, Christopher G. Brinton
Abstract
Task-wise federated multi-objective optimization (FedMOO) trains a shared model for competing prediction objectives under heterogeneous data, partial participation, and communication constraints. Existing methods commonly derive task weights from gradient or update geometry. This requires task-specific information or iterative server-side optimization. We introduce FedHV, which maps reference-relative objective slacks to closed-form inverse-slack weights. Each client optimizes one weighted loss and returns objective estimates with its model update. The protocol adds exactly 2m auxiliary scalars per participating client, yielding Theta(d + m) total per-client communication, compared with the Theta(md) task-specific communication of FSMGDA, and requires no additional synchronization stage. We analyze the resulting one-round-delayed weights under client heterogeneity, multi-step local updates, partial participation, and finite-sample objective reports. Under a fixed-horizon positive-slack reference condition, with the prescribed horizon-dependent step size and vanishing report error, FedHV achieves an O(T^(-1/2)) rate for the average squared log-hypervolume gradient norm; persistent report error determines the resulting stationarity neighborhood. The same bound controls the squared Pareto-stationarity residual. Across six Dirichlet-partitioned non-IID settings from four vision benchmark families and three training seeds, FedHV exceeds FSMGDA and FedCMOO in mean accuracy in five settings. Among these methods and uniform scalarization, it achieves the highest worst-task accuracy in four settings and improves the difficult CIFAR-10 objective in both CIFAR10-MNIST settings.