Decision models often misassign probabilities when several answers are correct
When Multiple Choices Are Correct: Asymmetric Probability Allocation in Jev and Other Decision Models
Artificial Intelligence
Summary
Sometimes, when computers decide between many good answers, they don't give each good answer a fair chance. The authors tested models that give probabilities to choices and found that even when all correct answers were ranked higher than wrong ones, some correct answers got less probability than expected. When the same correct answer was repeated multiple times, the models assigned very different probabilities to each copy. This means that models can rank good answers well but struggle to clearly identify all correct answers just by looking at their probabilities.
What this means in practice
- •For machine learning engineers: Improve decision systems by adjusting thresholds to better select multiple correct answers from probability scores in classification tasks.
- •For data annotation teams: Use caution when interpreting model probabilities as confidence for multiple correct labels during dataset labeling and quality assurance.
Authors
Rushab Munot
Abstract
Decision models return normalized probability distributions over candidate choices, which downstream applications can use to select among choices. Although their providers emphasize the reliability of these probabilities, calibration on tasks with a single correct choice does not establish their reliability when multiple choices are correct. We evaluate three decision models (Jev, Liquid d1, and GPT-6 Luna Decisions) on three diverse datasets (MetaTool, HotpotQA, and ECtHR) to study probability allocation and thresholding. Even when models rank every correct choice above every distractor, in 28% to 50% of those lists, at least one correct choice receives a probability below the uniform probability of 1/n, where n is the number of choices. We also present each model with five identical copies of a correct choice and observe probability disparities, with mean gaps ranging from 17 to 79 percentage points. We further show how these allocation disparities affect the recovery of multiple correct choices under probability thresholding. Decision-model probabilities can rank correct choices, but a probability threshold does not identify the set of correct choices.