Smote improves logistic classification when assumptions break down
PU classification under Non-SCAR: clustering-assisted logistic model with oversampling enhancement
Artificial IntelligenceMachine Learning
Summary
This study looks at a tricky type of data problem where you only have examples labeled as positive or unlabeled, and usual assumptions about the data don’t hold. The authors explore ways to improve how logistic regression models handle this challenge by using a method called SMOTE to balance the data and clustering to clean the labels. Their results from testing on many real and synthetic datasets show that adding SMOTE helps the models perform better when the usual data assumptions are violated. One of their logistic regression variants using regularization also showed some robustness in these cases.
What this means in practice
- •For machine learning engineers: Build better classifiers for datasets where only positive and unlabeled data are available and the typical random labeling assumption does not hold.
- •For medical data analysts: Improve patient disease classification when negative cases are not explicitly labeled and data imbalance confounds conventional modeling.
Authors
Konrad Furmańczyk, Kacper Paczutkowski
Abstract
This study addresses the PU classification problem under violations of the SCAR assumption. We investigate logistic regression-based approaches, namely the cluster method and its extensions with strict and non-strict Lasso regularization. The primary contribution of this work is the integration of the SMOTE technique to alleviate class imbalance and systematically assess its impact on the performance of the considered algorithms. SMOTE is first applied to rebalance the training dataset. Next, cleaning labels are derived via 2-means clustering. Logistic regression is then trained on the cleaned data, where identified positive instances are augmented with additional true positives and the remaining observations are treated as negative. The experimental evaluation is conducted on 13 real benchmark datasets and one synthetic dataset. For comparison, we include the naive approach and the Spy-EM method. The results demonstrate that incorporating SMOTE improves classification performance when the SCAR condition is violated and indicate moderate robustness of the LassoJoint method in this setting.