Prompt wording changes cause instability in AI model judging
Prompt Lottery in LLM-as-a-Judge: Measuring and Mitigating Sensitivity to Syntactic Rephrasing
Computation and LanguageArtificial Intelligence
Summary
The paper finds that when large language models judge other AI models, small wording changes in the judging prompts can cause big differences in scores and rankings. The authors call this the Prompt Lottery effect because results seem to depend on which prompt variation is used. They show that these models are much less consistent than humans when given slightly different prompt wordings. To fix this problem, they combine judgments from many prompt versions to get more stable and reliable evaluations.
What this means in practice
- •For ai evaluation teams: Use multiple prompt formulations to make AI model evaluations more consistent and reliable.
- •For ai service developers: Improve AI-powered content moderation by aggregating judgments from various prompt phrasings for stable decisions.
Authors
Ilija Subasic, Brett Levert, Andrew Rabinovich, Sajjad Pakdamansavoji, Zhao Chen
Abstract
LLM-as-a-Judge has become a standard for evaluating large language models, yet most approaches implicitly assume that a single judge prompt functions as a stable evaluation instrument. Prior work has primarily explored deliberate prompt modifications such as scoring rubrics, role descriptions, or chain-of-thought reasoning in order to optimize accuracy. In contrast, we investigate whether validated prompt variants intended to preserve evaluation semantics while differing in surface realization (\textit{paraphrastic rephrasings}) yield consistent judgments under a fixed evaluation task. We introduce \textit{Prompt Lottery}, a phenomenon in which paraphrastic rephrasings produce different LLMaJ judgments. Across a large-scale study spanning diverse evaluation tasks, judge families, and model backbones, we find that these variations can alter evaluation outcomes and, in some settings, reverse relative model rankings. A user study with 10 human annotators shows that LLMaJ judges exhibit 6-9$\times$ larger agreement degradation under prompt rephrasings than humans, falling below the generally accepted threshold in usability studies. This indicates that single-prompt evaluation is insufficiently robust for model ranking. Finally, we show that multi-prompt aggregation substantially improves robustness by emphasizing judgments that remain consistent across prompt variants.