Adversarial requests trick length predictions to jump scheduling queue

Jumping the Line: Exploiting Length Predictions in LLM Scheduling

Artificial Intelligence

Summary

Large language models often decide which user requests to handle first based on how long the response will be. The authors show that by adding special endings to their input, attackers can fool the system into thinking their responses will be shorter than they really are. This trick helps these requests get faster processing at the expense of others. The authors also test ways to reduce this problem by making length predictions less precise.

What this means in practice

  • For llm deployment teams: Improve fairness and robustness of LLM request scheduling by mitigating adversarial manipulation of length predictions.
  • For cloud service operators: Reduce scheduling delays by implementing coarse grouping of length predictions to defend against adversarial requests.

Authors

Yuyang Dai, Rana Shahout, Mahmood Sharif

Abstract

Efficient request scheduling is increasingly important for reducing completion time in large language model (LLM) serving. Size-based policies such as Shortest Job First prioritize shorter requests, but output lengths are unknown before generation, so practical schedulers rely on predicted lengths. We introduce JIL, an attack on prediction-based LLM schedulers that manipulates the scheduling signal to obtain higher priority and reduce completion time. Using TRAIL as a case study, JIL optimizes an adversarial suffix that causes a lightweight output-length probe to underestimate a request's length. We evaluate JIL on two datasets and four LLMs across varied request profiles and deployment configurations. JIL reduces predicted output lengths by up to 83.4 percent, and adversarial requests complete up to 1.53 times faster on average in end-to-end serving experiments. The reduction in predicted length is substantially larger than the change in actual output length, revealing a mismatch between the scheduler's estimate and the request's realized size. Response utility varies across models and tasks, exposing a trade-off between scheduling advantage and response quality. We also evaluate scheduler-side defenses and find that grouping length predictions into coarse intervals reduces JIL's scheduling advantage and mitigates delays to benign requests.