Post hoc method reveals feature contributions in black-box models

ONAM: Fitting Orthogonal Neural Additive Models for Post-Hoc Interpretability of Black-Box Predictions

Machine Learning

Summary

Many powerful AI models are hard to understand because their decisions come from complex processes. The authors introduce ONAM, a tool that breaks down AI predictions into parts linked to each input feature, making decisions easier to interpret. This method works after the model has made predictions and does not depend on the type of AI model used. Their approach ensures the breakdown is clear and mathematically well-defined, helping users see how each feature affects the outcome.

What this means in practice

  • For data science teams: Use ONAM to clarify complex model predictions by decomposing outputs into feature-specific effects across different AI models.
  • For healthcare analytics professionals: Apply ONAM to interpret AI predictions in healthcare, ensuring transparency in sensitive decision-making contexts without changing the original models.

Authors

David Köhler, Matthias Schmid

Abstract

Modern machine learning models, such as artificial neural networks, achieve high predictive performance through increasingly complex black-box architectures, limiting their use in settings where transparency, interpretability, and fairness are required. Here we present the R package ONAM, which provides a novel and practical implementation of functional decomposition, a technique to facilitate the interpretability of black-box models through structured effect representations. ONAM implements an algorithmic framework for the post-hoc decomposition of predictions into components associated with different input features. The package is designed for usability and efficiency, including functionalities for visualization as well as quantitative measures to assess interpretability. As a model-agnostic method, it can be applied to a wide range of prediction models. We describe the mathematical foundations, algorithmic design, and software architecture of ONAM. A central result of the proposed methodology is that the resulting decomposition satisfies the condition of hierarchical orthogonality, thereby guaranteeing that all components are identifiable and uniquely defined.