Evaluating reliability and costs of cleaning math training data
What Should We Remove? Evaluating the Reliability and Costs of RLVR Data Decontamination
Computation and LanguageMachine Learning
Summary
Math problem sets used to train AI that learns from rewards often overlap with test questions, which can cause unreliable evaluation. The authors studied different ways to clean these training sets to remove overlap, checking how often the cleaning mistakenly removes useful data. They found that common methods can falsely remove many training items, while an existing careful policy strikes a better balance between catching overlaps and avoiding false removals. Their work helps understand the trade-offs and risks when trying to clean data for testing fairness.
What this means in practice
- •For machine learning engineers: Improve training data cleaning methods to better balance test overlap removal without losing valuable data in math problem datasets.
- •For dataset curators: Assess and select data decontamination policies that reduce false positives in benchmark cleaning to maintain dataset quality.
Authors
Esther Xin
Abstract
Reinforcement learning with verifiable rewards (RLVR) math corpora are routinely screened for benchmark overlap, but the reliability and data-loss costs of cleaning policies remain underexamined. We audit sixteen open RLVR math corpora comprising 1,673,514 corpus-specific training items against six benchmarks containing 1,949 questions. We evaluate eight policies based on exact, lexical, and semantic matching using 594 independently and blindly dual-reviewed training-benchmark pairs. Calibration and held-out sets have disjoint benchmark questions and training-example families. A separate review of all 83 candidate matches for 80 sampled training items measures item-level false removal. At cosine threshold 0.80, 52 of 59 sampled removals (88%) were confirmed false removals; this is not a demonstrated population rate. No semantic threshold in our preregistered grid (0.80-0.98) meets the 90% conservative-precision target on calibration. The existing judge-adjudicated policy achieves 98.1% resolved precision and 98.2% candidate-frame recall on held-out corpora. It has zero confirmed false removals among six sampled removals, with one unresolved case; this does not establish zero risk. Within the evaluated candidate frame, this policy offers a more favorable precision-recall balance than the semantic-threshold alternatives tested. Recall does not measure contamination outside the candidate frame. We release reviewed labels, per-policy decisions, and coverage accounting to support evaluation of cleaning reliability and cost rather than overlap counts alone.