Automl tools differ in model variety and explanation stability
Equally Good, Yet Different: Benchmarking Rashomon sets in AutoML packages
Machine Learning
Summary
Some automated tools for building AI models find many different models that work equally well but explain their decisions in different ways. This can be risky because users might pick models based on their explanations instead of actual performance, a problem called x-hacking. The authors created a tool named ARSA ML to measure how diverse and stable these model sets are in popular AutoML packages. They found one tool, AutoGluon, tends to produce many diverse models with stable explanations, while another, H2O, makes fewer models that vary more in predictions and explanations, increasing the risk of x-hacking.
What this means in practice
- •For machine learning engineers: Assess and reduce risks of biased model selection by measuring model diversity and explanation stability in AutoML pipelines.
- •For data science teams in finance: Improve trustworthiness of automated binary classification models by identifying when AutoML tools produce unstable model explanations.
Authors
Katarzyna Woźnica, Katarzyna Rogalska, Zuzanna Sieńko, Mustafa Cavus
Abstract
The Rashomon effect describes the existence of multiple near-optimal models that achieve comparable performance while offering fundamentally different explanations. This creates a critical vulnerability in AutoML: x-hacking, the selective post-hoc choice of a model based on its explanation rather than predictive merit. No existing AutoML framework exposes this risk. We introduce ARSA ML, an open-source Python framework that quantifies Rashomon set structure and predictive multiplicity within AutoML pipelines. Using ARSA ML, we benchmark AutoGluon and H2O across 28 binary classification datasets, and conduct a post-hoc x-hacking analysis revealing a consistent structural asymmetry: AutoGluon produces larger, diverse sets with stable explanations, while H2O generates compact sets with markedly higher prediction divergence and explanation instability -- making H2O users considerably more exposed to x-hacking. This gap persists across all evaluated metrics and epsilon thresholds, pointing to a fundamental difference in each framework's model-building strategy. ARSA ML is available at https://pypi.org/project/arsa-ml/ .