Large language models can be tricked to waste more computing power
FragToken: Amplifying LLM Inference Costs through Noncanonical Token Generation
Cryptography and Security
Summary
Large language models (LLMs) are costly to run, so attackers try to make them use more resources. The authors found a hidden trick where the model can produce the same visible text but using many more internal steps. This means the model works harder without showing longer outputs, making attacks harder to detect. They created FragToken, a method to train models to do this token trick while still giving useful answers, proving this hidden attack is real and effective.
What this means in practice
- •For llm service providers: Detect and defend against covert attacks that increase inference cost without changing response length significantly.
- •For security teams: Assess supply-chain risks from models trained to produce resource-heavy token sequences to anticipate hidden denial-of-service tactics.
Authors
Zihan Wang, Rui Zhang, Xinyuan Qian, Qingchuan Zhao, Hongwei Li, Guowen Xu
Abstract
As large language model (LLM) inference becomes increasingly expensive, resource-consumption attacks pose a growing threat to model providers. Existing attacks typically amplify cost by inducing abnormally long or repetitive outputs on attacker-controlled or triggered requests, making them easier to detect and limiting their deployment-wide impact when benign traffic dominates. In this work, we uncover a previously overlooked token-level attack surface arising from the many-to-one mapping from token sequences to decoded text. Although standard LLMs predominantly generate the canonical token sequences induced by their tokenizers, the same text can also be represented by substantially longer non-canonical sequences. This representational flexibility exposes a new avenue for resource-consumption attacks: an attacker can train the model to favor such sequences, systematically increasing the number of autoregressive decoding steps without a proportional increase in visible response length. However, we empirically find that directly maximizing token fragmentation substantially degrades model utility, producing conspicuous answer-quality failures that undermine attack stealthiness. To address this challenge, we propose FragToken, a training-time framework that combines source-model self-distillation, capacity-aware filtering and budgeting, and BPE-Aligned Merging to induce fragmented generation under ordinary prompts while largely preserving model utility. We evaluate FragToken on four LLMs across three benchmarks. Across the four models, FragToken achieves a three-benchmark average token inflation ratio (TIR) ranging from 1.99 to 2.46, while causing only minor degradation in model utility. Our work reveals a covert LLM supply-chain threat that increases inference cost without requiring large volumes of attack requests while largely preserving utility.