Predictive value can change without losing information under recoding
Complementary Feature Domains: Information Preservation Does Not Imply Predictive-Contribution Preservation
Machine LearningArtificial Intelligence
Summary
This paper shows that keeping all information about a target does not guarantee that the usefulness of features for prediction remains the same. The authors explain that even when a transformation perfectly preserves information, it can still alter how well certain methods use that information to make predictions. They introduce a way to measure this change and demonstrate it with an experiment using heart signals. This finding helps clarify that information preservation and predictive usefulness are different concepts.
What this means in practice
- •For machine learning engineers: Improve model evaluation by understanding how data transformations affect prediction performance beyond information preservation.
- •For signal processing teams: Adjust feature representations in biosignal analysis to maintain prediction accuracy despite nonlinear recodings.
Tested on one dataset.
Authors
Timothy Oladunni, Farouk Ganiyu-Adewumi
Abstract
Complementary Feature Domains (CFD) theory characterizes predictive value as a context-indexed contribution system induced jointly by representations and their realization family. We show that Shannon-information preservation does not imply preservation of this contribution system: an invertible representation transformation can leave target information unchanged while altering predictive contribution under a restricted decision family. We formalize the resulting transition through a CFD contribution defect that measures how contextual contributions change under controlled recoding. For bounded Lipschitz utility, we show that each coalition utility shift is bounded by the behavioral distance between the attainable action sets before and after recoding; consequently, every contextual contribution defect is bounded by the sum of the corresponding coalition incompatibilities. Exact behavioral closure yields invariance, while increasingly accurate compensation yields restoration. A controlled ECG experiment illustrates the mechanism: a nonlinear bijective recoding preserves the information in a frozen time-frequency representation but changes accuracy under a fixed affine learner; applying the exact inverse restores all tested coalition accuracies. The result separates information preservation from realization-dependent contribution and provides a quantitative transition law for multi-representation prediction.