Interpretable rule-based machine learning matches black-box accuracy
SR4-Fit: A Unified Interpretable Rule-Based Machine Learning Framework for Informative and Trustworthy Decision-Making
Machine LearningArtificial Intelligence
Summary
Complex machine learning models are often hard to understand and trust because they act like black boxes, giving predictions without clear reasons. The authors created a new method called SR4-Fit that makes simple, easy-to-read decision rules while still being accurate and reliable. They showed it works well in predicting elections and other tasks, finding patterns that other models might miss. This means we can have trustworthy predictions without losing performance.
What this means in practice
- •For political data analysts: Predict election outcomes using transparent and reliable demographic-based rule sets.
- •For financial risk analysts: Build interpretable models that produce stable, compact rules to explain credit or investment risks.
Authors
Shyam Sundar Murali Krishnan, Dean Frederick Hougen
Abstract
In many high-stakes applications, machine learning is dominated by black-box models that require post hoc explanations to justify their predictions. These explanations are often unreliable because they do not reflect the model's actual computations, limiting accountability and trust. A natural alternative is to use models that are interpretable by design. However, existing rule-based approaches, such as RuleFit and decision trees, while transparent, often lack stability and predictive strength, reinforcing a perceived trade-off between traditional performance measures and model understandability. To address this, we propose Sparse Relaxed Regularized Regression Rule-Fit (SR4-Fit), an intrinsically interpretable algorithm for both classification and regression that produces compact and stable rule sets without sacrificing performance. Using demographic data from the U.S. Census Bureau's American Community Survey, SR4-Fit predicts U.S. House election outcomes with high accuracy and interpretability while uncovering demographic interactions missed by black-box models. We further validate SR4-Fit across fourteen benchmark datasets (six classification and eight regression), where it outperforms existing rule-based methods, including RuleFit and decision trees in terms of accuracy, stability, and compactness while remaining competitive with black-box models in predictivity. These results demonstrate that interpretability and predictive reliability need not be mutually exclusive, offering a practical and transparent alternative for high-stakes decision-making.