Human centered AI should model multiple valid interpretations
Whose Ground Truth? Embracing Ambiguity in Human-Centered AI
Artificial IntelligenceMachine Learning
Summary
Many AI systems assume there is one correct answer when interpreting human-related data, but people often see the same thing in different, equally valid ways. The authors argue that instead of forcing AI to pick a single answer, systems should acknowledge and represent this ambiguity. This approach would better reflect how real people think and experience things, leading to AI that respects diverse perspectives. They suggest changes in how AI is designed, trained, evaluated, and governed to include multiple human interpretations rather than just one.
What this means in practice
- •For product designers: Design AI interfaces that reveal multiple user interpretations to better tailor experiences to diverse human perspectives.
- •For content moderators: Develop moderation tools that consider several plausible human judgments instead of a single binary decision, improving fairness and transparency.
A position paper. It proposes an approach and reports no results.
Authors
Jingyao Wu, Mohammad Tariqul Islam, Per Rådberg Nagbøl, Julie Gerlings, Giovanni Leoni, Georgiana Fryza, Lars Pilegaard Thomsen, Jakob Mainz
Abstract
As AI systems increasingly interact with people and make decisions about them, understanding human interpretations becomes an important part of developing human-centered AI. Conventional machine learning and AI systems are largely developed under the assumption that a single definitive ground truth exists, with variability in human annotations often resolved through aggregation or treated as noise. However, for many human-centered tasks, human interpretation is inherently ambiguous, and multiple interpretations of the same input may be simultaneously reasonable and valid. Reducing such ambiguity to a single target risks overlooking meaningful information about the diversity of human perception, judgment, and experience. In this position paper, we call for a shift towards modeling the interpretation space of plausible human judgments, while distinguishing meaningful ambiguity from annotation noise. We argue that this perspective should guide how AI systems are represented, learned, evaluated, deployed, and governed, supporting more human-centered AI that better reflects the diversity of human interpretation.