Language model rubrics struggle to resist tricky impossible questions
ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals
Machine LearningComputation and Language
Summary
Automated rubrics generated by AI are used to judge answers, but they can be tricked by responses designed to fool them. The authors studied tasks that are impossible to answer honestly because the questions push for unsupported conclusions. They created a benchmark to test how well rubrics can spot honest from dishonest answers and found many rubrics get fooled often, especially when they focus on the wrong specific criteria. Surprisingly, simple generic rubrics sometimes perform better at resisting trick answers than ones tailored to each task.
What this means in practice
- •For automated grading systems: Improve rubric design by testing them against impossible tasks to ensure they reward honest answers correctly and resist adversarial cheating.
- •For natural language processing engineers: Develop more robust language model evaluation tools that avoid misleading signals caused by overly specific task rubrics.
Authors
Bowen Qin, Yi Xie, Yesheng Liu, Xi Yang
Abstract
Language model-generated rubrics are increasingly used as reward signals for rubric-based reinforcement learning, LLM-as-a-judge evaluation, and automated grading. Such rubrics are reliable only if they reward honest answers over adversarial answers optimized to exploit them. Yet their robustness to such optimization remains poorly understood. We isolate the hardest regime: impossible tasks, where the prompt pressures the model toward an unsupported conclusion, so the only honest response is to acknowledge the impossibility. We introduce ImpossibleRubrics, a benchmark of 169 impossible tasks spanning six impossibility categories, each paired with a verifiable oracle certificate specifying what an honest answer may and may not claim, together with 48 answerable controls. Rather than providing fixed rubrics, ImpossibleRubrics provides task environments and certificates, allowing rubrics to be generated downstream and then adversarially tested for whether they reward certificate-violating answers. Eleven generators are exploited 8--26% of the time on the unbiased 150-of-169 environment cut; on a deliberately selected stress cut the strongest generator we measured is still exploited 36% while a certificate-faithful rubric is exploited 0%, so what we measure is a rubric-quality gap, not task impossibility. One result runs against intuition. A single generic rubric ("be decisive, penalize hedging") used unchanged for every task is exploited 64% of the time, and seven of the eleven generators are exploited more often than that while writing a rubric tailored to each one. The tailored criteria appear to tell an attacker which claim to fabricate. The problem is not that rubrics are vague; it is that they are specific about the wrong things.