CRISP method cuts training data drastically while preserving fraud detection accuracy
CRISP: Scalable Importance-Stratified Coresets for Imbalanced Tabular Learning
Machine Learning
Summary
Large datasets with many more normal examples than rare ones make training machine learning models slow and expensive. The authors present CRISP, a new way to smartly pick a much smaller subset of data while keeping most of the model’s accuracy. This method keeps very few normal examples but chooses them carefully based on how important they seem, allowing faster training with almost the same results. They tested CRISP on real fraud and advertising data, showing big savings in training time without losing performance.
What this means in practice
- •For fraud detection teams: Reduce training time for fraud detection models on massive transaction datasets while retaining high accuracy with fewer normal examples.
- •For advertising analytics teams: Speed up training of click or conversion prediction models on large imbalanced ad datasets by selecting smaller representative subsets.
Authors
Hardhik Mohanty, Indrayana Rustandi, Mohamadreza Sheibani
Abstract
Large imbalanced tabular datasets make repeated gradient-boosted tree training expensive. Existing coreset methods often lose accuracy when most majority examples are removed. We present CRISP (Coreset Reduction via Importance-Stratified Pruning), a linear-time method that allocates a negative-class budget across quantile strata of a proxy-model score. Sample weights account for unequal inclusion probabilities. At 95% negative-class reduction on a production fraud dataset, CRISP trains on approximately 1.70M of 25M rows and retains 99.7% of full-data Average Precision. This is a 93.2% reduction in total training rows. On public CriteoPrivateAds, CRISP has the highest mean Average Precision at each tested rate from 90% to 99.4% majority reduction. Sparkov results are mixed at lower rates, but CRISP has the highest mean at 99.2% and 99.4%. Ablations identify budget allocation and inverse-propensity weighting as the main sources of the production-dataset gain.