Matthews correlation coefficient reviewed for fairer binary classification
The Matthews Correlation Coefficient: Fifty years
Artificial Intelligence
Summary
Measuring how well a computer tells two things apart can be tricky, especially when one thing happens much more often than the other. The Matthews Correlation Coefficient (MCC) has been used for 50 years to help with this, but it isn’t perfect and sometimes misunderstood. The authors explain what MCC really means and how it behaves when data is unbalanced. They also create new tools to understand similar scores better, helping teams pick the right measure for accurate and fair results.
What this means in practice
- •For machine learning engineers: Evaluate and select classification metrics better for imbalanced datasets to make predictions more reliable and fair.
- •For medical data analysts: Use clarified MCC properties to improve the fairness and accuracy of diagnostic test evaluations involving rare diseases.
A survey. It maps existing work.
Authors
Jesus S. Aguilar-Ruiz
Abstract
Fifty years after its introduction, the Matthews Correlation Coefficient (MCC) remains central -and debated- in binary classification. This survey offers a comprehensive retrospective and an in-depth analysis of MCC, covering its historical development, mathematical foundations, and implications for practice. We clarify long-standing misconceptions, formalize its equivalences (to Pearson's correlation and to Chi2 in 2x2 tables), and examine its behavior under class imbalance. Beyond MCC, we develop analytical (derivatives) and graphical (contour plots) frameworks that generalize to any confusion-matrix-based metric. These frameworks expose structural properties -bias, robustness, sensitivity- that scalar summaries conceal, and they provide actionable guidance for selecting and interpreting performance measures in applications where reliability and fairness matter. Together, our results position MCC within a broader toolkit while clarifying when it is informative -and when alternatives may be preferable.