Framework quantifies value of new data beyond existing assets
The Data Differentiator Framework (DDF): Quantifying Marginal Data Value with Information-Theoretic and Game-Theoretic Measures in an Agentic Evaluation Pipeline (SDSAE)
Machine Learning
Summary
In companies with lots of data, figuring out if buying new data is worth it can be hard. The authors created a method called the Data Differentiator Framework (DDF) that checks if new data really adds useful info beyond what is already owned and if it justifies the cost. Their method looks at how much new information the data brings, how much it costs, and business priorities like trust and data quality. They tested this on a fraud detection dataset and showed it successfully identifies valuable new data while filtering out redundant ones.
What this means in practice
- •For enterprise data teams: Decide whether to invest in new data assets by quantifying the value they add beyond existing datasets and their cost.
- •For fraud detection teams: Improve fraud models by selectively acquiring data proven to add meaningful performance gains validated statistically.
Tested on one dataset.
Authors
Sukesh Kumar, Dilip Singh
Abstract
In data-rich enterprises the hard question is not whether a candidate dataset is good in isolation, but whether it adds information beyond what the organization already holds, and whether that increment is worth its price. We present the Data Differentiator Framework (DDF), a composite scoring method for candidate data assets that combines (i) a mutual-information-based redundancy penalty against the existing portfolio, (ii) a cost-normalized, significance-gated marginal utility score derived from an ablation study, and (iii) explicit governance inputs (trust, identity resolution quality, strategic weights) set by the business. The framework is executed by a staged pipeline, the Strategic Data Sourcing Agent Ecosystem (SDSAE), that short-circuits expensive model-retraining steps for assets failing governance or redundancy gates. Conceptually, the DDF extends the relevance-minus-redundancy principle of mRMR feature selection from the feature level to the asset level, and adds cost normalization, a statistical significance gate, and governance terms that per-sample valuation methods such as Data Shapley do not model. We validate the framework on the public credit-card fraud benchmark (284,807 transactions) with five constructed candidate assets spanning the redundancy spectrum. Only the genuinely differentiated asset passes the significance gate ($Δ$AUC = +0.0029, DeLong $p = 1.4\times 10^{-9}$, Holm-corrected $p = 7.2\times 10^{-9}$) and receives an acquire recommendation (DDS = +0.663); the fully redundant asset scores lowest (DDS = -0.135). The recommendation is stable across 19 of 22 admissible strategic-weight configurations and across all decision thresholds in [0.15, 0.66]. We state clearly which quantities in the case study are computed from data and which are simulated governance inputs, and we discuss the design limits of a single-dataset evaluation.