DeepSeek accelerates long-context searches with efficient attention reuse
RedKnot-MLA: Multi-Head Offline-Online Reuse for DeepSeek-V4 Long-Context Serving
Artificial Intelligence
Summary
Longer text understanding in AI systems can be slow and require lots of memory. The authors improved a method called multi-head latent attention to reuse parts of previous computations, making the process faster and more efficient without breaking the usual rules each part expects. They split the work into offline (preparing documents before queries) and online (adjusting during queries) steps, merging results in a smart way that saves memory. Their system shows better speed and accuracy in tests, although some cases see smaller improvements. They also explain how their technique can work with other advanced AI features and the limits of their measurements.
multi-head attentionlatent representationoffline processingonline servingRoPE (rotary position embeddings)tokenlocal and global attention headsspeedupF1 scoresparse mixture of experts (sparse-MoE)
Authors
Yang Liu, Zhaokai Luo, Huayi Jin, Ruozhou He, Chenchen Hong, Mingxiao Ma, Biao Zhang, Zhiyong Wang, Boyu Wang, Guanjie Chen, Yifei Liu, Tao Xie, Junhao Hu
Abstract
Multi-head latent attention (MLA) exposes many logical query heads through one packed latent KV stream. This representation is memory efficient, but it removes the physical per-head cache boundary assumed by conventional head-wise reuse. We present our system, a DeepSeek-V4 realization of RedKnot's head-aware reuse principle. Each immutable document is processed offline at canonical position zero; certified Local-head contributions are retained as MLA-Off. At serving time, query-side RoPE relocation restores the document's request position, a small Global-head set and protected Local token rows are recomputed as MLA-Online, and the two paths are merged before a single shared output projection. The packed MLA latent is never split. DeepSeek-V4-Flash uses 37 reusable layers and a 56/8 Local/Global partition, giving a 75.29% analytic logical head-row ceiling; the Pro-0813 profile uses 55 layers and 112/16 heads, giving 78.89%. Frozen Flash operating points show hot-artifact TTFT speedups of 2.02-3.84x. At 256K, the archived three-dataset study reports an aggregate F1 change of +3.24 percentage points, an EM change of +4.16 points, and a 78.7-79.5% analytic major-operator arithmetic saving, while one dataset decreases by 2.81 F1 points. A separate author-reported 256K hot-artifact QPS measurement is approximately 2.0x; because its raw concurrency trace is not included in this bundle, we mark it as preliminary rather than archived evidence. We describe the factorization, position repair, token-row closure, sparse-MoE support, TP8 integration, and the measurement boundaries needed to interpret these results.