Provider-side attacks inflate large language model outputs and costs
The More It Says, the More You Pay: A Black-Box Audit of Provider-Side Token Inflation in LLM Services
Cryptography and Security
Summary
In some language model services, users pay based on how many words the model generates. The authors found that some providers might secretly increase the length of responses to make users pay more, without clearly changing helpfulness. They designed several ways this can be done and showed it’s practical. They also created a simple test that can detect when such manipulations happen, even without knowing the original clean response. When tested on multiple models and real services, this test flagged several services likely inflating output length.
What this means in practice
- •For llm service providers: Detect manipulations that inflate user costs by covertly increasing output token length during generation.
- •For api security teams: Audit black-box language model APIs for hidden token inflation attacks without needing trusted reference models or historical data.
Authors
Leilei Chen, Lan Zhang, Chen Tang, Pengcheng Sun, Jiewei Lai, Yixiao Huang, Zhaopeng Zhang, Xinpeng Shen
Abstract
In pay-per-token LLM services, the more a model says, the more users pay. Dishonest providers can covertly manipulate generation to inflate output tokens while largely preserving task utility. We define such manipulation as a Provider-Side Token Inflation Attack (PTIA) and instantiate five representative attacks at the query, prompt, representation, and model levels of the provider-controlled pipeline. Our experiments show that each attack increases mean output length to more than 10.2x the clean baseline, demonstrating PTIA's financial appeal and feasibility at multiple stages of generation. Yet auditing PTIA from black-box responses is difficult for users. Our key observation is PTIA saturation: an initial attack sharply lengthens output, but further strengthening or composition has much less effect. We trace this saturation to stopping behavior: an initial PTIA sharply lowers the end-of-sequence token probability, whereas further intervention lowers it only marginally. Building on this insight, we design a lightweight single-probe audit that applies a controlled lengthening intervention. Under PTIA, the probe induces far fewer additional tokens than under normal service. The audit requires neither a trusted local reference model nor historical clean responses, and its separately issued original and probed requests resemble ordinary traffic, making evasion difficult. Across four open-weight models, it achieves an average detection rate of 85.1% with false-positive rates below 2%. Across 15 real LLM API services, the audit flags 7 for PTIA-consistent behavior.