DSpine improves parallel speculative decoding for longer accepted continuations
Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding
Machine LearningArtificial Intelligence
Summary
Generating multiple word predictions at once in language models can cause inconsistencies that shorten the trusted output. The authors show that better communication between nearby word predictions during the model’s layers can help fix this. They propose DSpine, which lets each predicted word influence the next at every step inside the model, improving accuracy and speed. This approach was tested on math, coding, and chat tasks and showed longer consistent output and faster processing than previous methods.
What this means in practice
- •For language model engineers: Increase the length of reliable outputs in large language models by integrating causal conditioning at each model layer during parallel decoding.
- •For ai service providers: Speed up response generation for AI-powered chat and coding assistants by using DSpine's efficient parallel decoding to improve throughput.$Commercial implications: Enables AI platform providers to deliver faster and more coherent language generation services.
Authors
Haohui Zhang, Keyu Chen, Haocheng Sun, Weibo Gu, Ruizhi Qiao, Xing Sun, Bo Jiang
Abstract
Parallel speculative drafting generates multiple candidates in one backbone pass, but independent token selection can produce inconsistent continuations that shorten the accepted prefix. Existing methods mostly leave conditional decoding to a lightweight module after the backbone, which limits the flow of predecessor information to successors. Our analysis of DFlash shows that early positions already form recoverable predictions in shallow layers, and that accurate adjacent predecessors help successors more when they enter earlier. We therefore propose DSpine, a drafter with causal conditioning injection throughout the backbone: at every layer, gated adjacent injection writes each predecessor's predicted feature into its successor, so the causal conditioning chain unfolds over network depth while all positions update in parallel. A unified transfer space built from the target model's output embeddings unifies layer-wise injection with predecessor-conditioned decoding, and layer-wise output-embedding supervision promotes the formation of predicted features in shallow layers. Fused kernels and a transition cache execute both efficiently in parallel within SGLang. Across seven math, code, and chat benchmarks, DSpine achieves the longest acceptance length at both temperatures on Qwen3-4B and Qwen3-8B. At temperature zero on Qwen3-8B, it raises the seven-benchmark mean from DFlash's 3.77 to 4.82 (+27.8%); in SGLang serving tests, it delivers 23.3% higher throughput than DFlash on average.