Personalized federated fine-tuning reduces medical language model costs
PFedQFT: Personalized Federated Quantum-Inspired Fine-Tuning for Medical Large Language Models
Artificial Intelligence
Summary
Medical data is stored separately by different hospitals and cannot be easily shared because of privacy and rules. The authors created a new way called PFedQFT that helps hospitals work together to improve large language models without sharing raw data. Their method reduces the amount of data and computing power needed by reusing certain parts of the model and customizing for each hospital's needs. Tests on medical tasks showed that this approach saves resources while keeping performance high.
What this means in practice
- •For hospital data teams: Collaborate across institutions to fine-tune medical language models efficiently without sharing patient data by using compact shared adapters with personalized adjustments.
- •For enterprise ai developers: Implement resource-efficient federated fine-tuning techniques for customizing large language models in privacy-sensitive domains using quantum-inspired orthonormal adaptation methods.
Authors
Qihan Huang, Lingxia Zhang, Guye Zhang, Zizhu Wang, Xiaowei Mao
Abstract
Medical data remain fragmented across institutional silos because of privacy, governance, and ownership constraints. Federated learning enables medical institutions to adapt large language models (LLMs) collaboratively without transferring raw records. However, independently parameterized low-rank adapters incur repeated parameter and communication costs across Transformer projections, while uniform aggregation may limit adaptation to heterogeneous institutional data. We propose PFedQFT, a quantum-inspired framework for personalized federated fine-tuning with a compact shared adapter. Specifically, PFedQFT constructs reusable orthonormal adaptation bases from compact skew-symmetric generators using a truncated matrix-exponential approximation followed by QR orthonormalization. Compatible projections reuse these bases, while a globally shared full core preserves the flexibility of each projection. Each client further maintains a lightweight private residual operator, whose directions are learned globally and whose coefficients remain local. Experiments on seven public medical tasks and multiple LLM backbones show substantial reductions in trainable parameters and per-round adapter upload volume, with competitive predictive performance on most tasks.