An Efficient and Privacy-Preserving Architecture for Cross-Institutional Collaborative RAG
2026-05-25 • Cryptography and Security
Cryptography and SecurityDistributed, Parallel, and Cluster Computing
AI summaryⓘ
The authors developed FedRAG, a system that helps large language models access shared knowledge from different organizations without revealing private data. They solved a tricky problem where traditional methods require sharing sensitive information across computers, which breaks privacy rules. Their new method scrambles data so calculations can be spread out safely, avoiding delays and extra security costs. Tests show FedRAG keeps answers nearly as good as usual but works much faster than other secure systems.
Retrieval-Augmented Generationfederated learningTransformer self-attentionprivacy-preserving computingdistributed inferencedata silossecure multi-party computationfeature scramblingknowledge base integrationlatency reduction
Authors
Chenxin Mao, Shangyu Liu, Zhenzhe Zheng, Fan Wu, Jie Wu, Guihai Chen
Abstract
Retrieval-Augmented Generation (RAG) empowers LLMs with external knowledge, making cross-institutional domain-specific knowledge base integration a highly promising deployment paradigm. Despite this potential, strict privacy regulations create severe "data silos" that obstruct such collaboration. Building federated RAG systems requires distributed inference, but the Transformer's self-attention mechanism fundamentally conflicts with this by mandating cross-node access to distributed Key-Value caches. To address this challenge, we present FedRAG, a high-throughput, privacy-preserving federated RAG framework. At its core is a novel Scrambled Distributed Attention protocol that utilizes numerically stable feature scrambling and token permutation. By dynamically delegating scrambled computations to collaborating nodes, our system successfully decouples attention execution from data localization without exposing plaintext. Crucially, our approach requires no specialized hardware or model retraining, circumventing the prohibitive latency and communication overheads of cryptographic solutions while robustly defending against intermediate state inversion attacks. Extensive evaluations demonstrate our framework preserves negligible (<0.1\%) model utility degradation and achieves up to a 62$\times$ latency reduction over existing secure baselines, sustaining practical, human-reading throughput for cross-institutional knowledge synergy.