SplitLite: Low-Rank Residual Compression for Split Learning
2026-08-24 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors address the problem of heavy communication when fine-tuning large language models on devices via split learning, which splits work between devices and a powerful server. They found that the changes in data passed back and forth during training have a special low-rank structure. By using this pattern, their method, SplitLite, sends only compressed, important parts of the data, drastically cutting communication costs. Tests showed this method saves a lot of data transfer without losing model accuracy.
federated learningsplit learninglarge language modelsLoRAactivationgradientsingular value decompositionlow-rank approximationcommunication efficiencyfine-tuning
Authors
Tao Li, Yulin Tang, Qi Guo, Xianhao Chen
Abstract
Federated fine-tuning of on-device large language models (LLMs) faces a significant computing burden. To overcome this limitation, split learning (SL) has emerged as a promising solution, which offloads the primary training workload to a powerful server. However, SL requires exchanging high-dimensional activations and gradients between clients and the server, resulting in prohibitive communication costs. To overcome this challenge, we propose SplitLite, a communication-efficient split federated LoRA fine-tuning method that exploits the low effective rank structure of consecutive-epoch activation and gradient residuals. Our key finding is that, when LoRA uses rank $r$ updates in parameter space, the activation and gradient residuals of the same data sample between adjacent epochs also exhibit effective rank-$2r$ and rank-$4r$ structures, respectively. By revealing this property, SplitLite transmits only quantized truncated singular value decomposition (SVD) residual factors, thereby significantly reducing both activation uplink and gradient downlink traffic. Extensive experiments on the GLUE benchmark across a series of advanced on-device LLMs demonstrate that our method reduces activation uplink communication costs by up to 93.5\% and total communication costs by up to 83.7\%, without performance degradation.