AI summaryⓘ
The authors explore a new way, called Structured Newton Layer Parallelism (SNLP), to speed up and improve encrypted Transformer model computations under fully homomorphic encryption (FHE). Normally, these models are slow because many nonlinear steps must happen one after another, but SNLP reduces how many of these steps happen sequentially. Their tests show that SNLP cuts down the number of complex encryption refreshing steps significantly, with only a small drop in accuracy. They also find that most errors come from approximating the softmax function, and that the typical arithmetic noise in their setup is negligible. This means SNLP helps make encrypted Transformers more practical and works well alongside other methods that focus on making operations easier for encrypted data.
Fully Homomorphic Encryption (FHE)Transformer ModelsStructured Newton Layer Parallelism (SNLP)Softmax ApproximationPolynomial ApproximationBootstrappingCKKS Encryption SchemePerplexityNonlinear DepthError Amplification
Authors
Ligong Han, Kai Xu, Hao Wang, Ruijiang Gao, Akash Srivastava
Abstract
Fully homomorphic encryption (FHE) enables computation on encrypted data, but practical encrypted Transformer inference is bottlenecked by the sequential composition of many nonlinear blocks. We study whether Structured Newton Layer Parallelism (SNLP) can make this inter-layer composition more FHE-friendly: each Transformer block still requires polynomial approximations for operations such as softmax and RMSNorm, but SNLP reduces the layerwise sequential nonlinear depth from L stages to a small number of solver iterations plus linear structured corrections. Using a simulation framework based on Chebyshev polynomial approximations, we measure error accumulation under sequential versus SNLP inference across 8 models and 4 architecture families. On a 0.5B IDN-trained model, SNLP reduces symbolic bootstraps from 53 to 20 (2.65x) with only +1.2% perplexity degradation, while lowering error amplification (1.36x vs. 1.42x). Across all tested models, SNLP has lower amplification than sequential inference. Ablations show that softmax approximation dominates the error budget and CKKS arithmetic noise is negligible in our setting, suggesting that SNLP is complementary to block-level FHE-friendly operator design rather than a replacement for it.