Speculative decoding vulnerabilities increase AI model inference costs
Speedbumps: Rejection Attacks on Speculative Decoding
Cryptography and SecurityArtificial Intelligence
Summary
Speculative decoding is a way to make large language models faster by guessing several words at once before checking them carefully. The authors found new kinds of attacks called Speculative Rejection Attacks that add tricky endings to inputs, causing the fast guessing to fail more often. This forces the model to do extra work, making it slower and more expensive to use. Their attacks can sometimes make the fast method slower than the regular way and affect the quality of the model's output.
What this means in practice
- •For ai service providers: Identify and guard against inputs that increase computational costs through speculative decoding attacks to maintain inference speed and reduce expenses.
- •For cybersecurity teams: Detect adversarial input patterns designed to degrade language model performance and avoid potential denial-of-service style resource exhaustion.
Authors
Adam Y. J. Jones, Yu Yuan, Sergio Maffeis
Abstract
Speculative decoding is a popular technique for increasing the speed and reducing the costs of large language model (LLM) inference by verifying multiple draft tokens in a single target-model forward pass. The resulting benefit depends on the ability of the drafter to approximate the target model's distribution. In this work, we study Speculative Rejection Attacks (SRAs), a novel class of attacks that cause draft and target models to disagree more often, resulting in fewer draft tokens being accepted per draft cycle. This leads to more target model forward passes needed per generated token, slowing down inference and increasing costs for the victim. We introduce two attacks which append an adversarial suffix to attacker-controlled content to degrade speculative decoding on a victim's prompts. Both attacks optimise the expected length of the accepted speculative prefix, estimating per-depth acceptance from the target's probability of the drafted proposals (Speedbump-P) or from the overlap between the draft and target distributions (Speedbump-D). In some cases, attacks degrade speculative decoding to the point of being slower than autoregressive decoding. The degradation reduces the output quality - regularisation restores output quality but gives up most of the degradation, trading effectiveness for stealthiness. Additionally, the suffixes remain effective under sampling, and transfer across drafters (Speedbump-P) or across target models sharing a drafter (Speedbump-D). These findings identify the draft-target interaction of speculative decoding as a realistic attack surface through which adversarial inputs can inflate inference costs.