DPara speeds up AI text generation by overlapping key decoding steps
When Parallel Drafter Meets Parallel Speculative Decoding
Computation and Language
Summary
Generating text with AI models often involves processing steps that happen one after the other, which slows things down. The authors present DPara, a new method that overlaps these steps by preparing multiple options in advance without guessing wrong. This approach avoids costly restarts and speeds up text generation significantly. Tests show DPara is faster than previous methods on math, coding, and chat tasks.
What this means in practice
- •For ai model developers: Integrate DPara to reduce latency in large language model text generation while maintaining accuracy.
- •For cloud service providers: Deploy DPara to improve throughput and speed of AI-powered customer support chatbots across multiple users.
Authors
Fuliang Liu, Xue Li, Kun Qian, Zhibin Wang, Wanchun Dou, Wenyuan Yu, Chen Tian
Abstract
DSpark-style parallel drafters have made speculative decoding highly effective, yet their draft phase remains serialized on the critical path of every round. Parallel speculative decoding (PSD) overlaps drafting with verification, yet existing methods must guess the accepted prefix and bonus token in advance: a wrong guess reverts the whole batch to serial drafting. We present DPara, a PSD framework that reuses effective parallel drafters yet guarantees backbone--verification overlap in every round, thereby eliminating this probabilistic fallback altogether. While the target verifies, DPara's diffusion backbone precomputes draft representations for every acceptance boundary with the bonus left unspecified; a lightweight autoregressive head then combines the revealed verification outcome with the matching precomputed representation to emit the next round's draft tokens almost instantly---fully parallelizing the dominant backbone forward with verification and leaving only the negligible head cost serial. Experiments on Qwen3-8B and Qwen3-14B across seven math, coding, and chat benchmarks show that DPara achieves average speedups of $3.21\times$ and $3.52\times$ over autoregressive decoding, surpassing the strongest serial and parallel speculative decoding baselines alike.