Federated fine tuning improves personalized model adaptation accuracy
FedLAFP: Low-Rank Aggregation Meets Full-Rank Personalization in Federated Fine-Tuning
Artificial Intelligence
Summary
When many devices train a shared AI model without sharing their private data, it’s hard to create one version that works well for everyone. The authors propose FedLAFP, a new way to combine a common part shared across all devices with a private part each device can customize fully. This method helps devices learn both shared knowledge and personal preferences more effectively. Tests show this approach improves accuracy on various image recognition tasks over previous methods.
What this means in practice
- •For mobile app developers: Enable mobile apps to personalize AI models locally while benefiting from shared improvements without sending raw data.
- •For enterprise ai teams: Improve personalization of federated AI models across diverse company devices with lower communication overhead and better accuracy.
Authors
Mengjun Yi, Huaian Gu, Yinghao Ai, Furao Shen, Jian Zhao
Abstract
Federated parameter-efficient fine-tuning enables clients to adapt pre-trained models without sharing raw data or communicating the full model, but statistical heterogeneity makes a single global adapter insufficient for personalized prediction. Existing personalized methods typically use the same low-rank structure for both shared and private adaptation, overlooking their distinct requirements for aggregation and personalization. We propose FedLAFP, a role-aware framework that couples a compact, globally aggregated LoRA branch with a client-private, full-rank-capable RandLoRA branch. The shared branch provides an efficient interface for transferring common knowledge, whereas the private branch combines fixed random low-rank bases with learned scaling coefficients to provide expressive client-specific adaptation without additional communication. Client- and layer-specific mixing coefficients jointly fuse the two branches, and only the shared LoRA parameters are exchanged. A controlled linear study supports this role assignment: LoRA yields more aligned client updates and lower aggregation error, while RandLoRA more accurately recovers client-specific residuals. Experiments across four visual recognition benchmarks show that FedLAFP consistently outperforms local-only and federated LoRA baselines, achieving an average personalized accuracy of $86.93\%$ and exceeding the best baseline average by $1.30$ percentage points.