Probabilities explained as outcomes of prediction methods
A Unifying Perspective on Probabilities as Model Predictions
Computers and SocietyMachine Learning
Summary
Probabilities are everywhere, but people disagree on what they really mean. This paper explains that all probabilities come from methods that predict events using models, showing that even so-called objective probabilities depend on assumptions. The authors also show when probabilities can reliably guide decision-making by meeting a criterion called finite calibration. Their work links different views on probability and clarifies when acting on probabilities leads to good outcomes.
What this means in practice
- •For data scientists: Choose and interpret probabilistic models knowing all probabilities depend on assumptions and model calibration ensures useful decision guidance.
- •For software engineers: Design prediction systems with calibration tests to ensure probability outputs reliably support decision-making on finite event sets.
A position paper. It proposes an approach and reports no results.
Authors
Benedikt Höltgen
Abstract
Although probabilistic statements are ubiquitous, foundational disagreements persist about their understanding, as exemplified by debates between Bayesians and frequentists; moreover, it is unclear when and why acting on them actually leads to desirable outcomes. Here, we argue that every probability is the output of a \emph{prediction method}, that is, it depends on both a particular way of constructing abstractions and a way of transforming them into predictions. Through this, we provide a unifying perspective on supposedly different kinds of probabilities and show that even supposedly objective ones are model-dependent. We demonstrate that when a finite calibration criterion is met, one can anticipate the distribution of utilities for a given policy and inform successful decision-making on finite sets of events. Based on the notion of prediction methods, inductive arguments, and the probability calculus, we explain the feasibility of the calibration criterion in many settings. Overall, we develop a coherent perspective on probabilities and their use, connecting key intuitions behind other interpretations along the way.