Limits of fixing data shifts with linear maps under noise and distortion
Detecting a Shift Is Not Enough: Exact Minimax Limits of Linear Representation Repair
Machine Learning
Summary
Sometimes data collected from two different sources can have a shift that is easy to find but hard to fix without changing the data’s important features. The authors study how well we can correct this kind of shift using a linear transformation that works on both data sources without causing too much distortion. They find an exact formula that tells how much shift can remain after correction depending on data dimensions and signal quality. Their results show that detecting the shift is much easier than fully removing it, and they validate this on real EEG data where more recordings eventually stop improving correction.
What this means in practice
- •For clinical data teams: Predict the effectiveness of linear shift corrections in EEG and similar clinical signals to decide if more data or better methods are needed.
- •For wearable device engineers: Determine how many paired device calibrations are required to reliably reduce participant differences causing data shifts in wearable sensor outputs.
Tested on one dataset.
Authors
Anuar Aimoldin, Yankai Chen, Ayana Mussabayeva, Xue Liu
Abstract
A mean shift between two data sources can be easy to detect but hard to remove without substantially changing their representations. We cast its removal as a statistical decision problem: from noisy differences between paired calibration measurements in $\mathbb{R}^d$, learn one linear map, applied to both sources under a hard distortion budget, that leaves as little of the shift as possible on fresh data. We derive the exact finite-sample minimax risk over all such maps, $(d-k) \mathbb{E}[1/(d+2J)]$ with $J\sim\mathrm{Pois}(κ/2)$, where the budget allows deleting $k$ directions and $κ$ is the calibration signal-to-noise ratio. Projecting out the mean calibration difference attains it without knowing $κ$ or the noise scale. This exposes a detection-repair gap: detecting the shift needs only $κ\gg\sqrt d$, whereas removing a fixed fraction of it at constant distortion needs $κ\asymp d$, as for estimating its direction. Standard linear concept erasers (MP, SAL, LEACE) remove the same calibration difference, so the formula gives, before fitting, exactly how much shift they leave on fresh data and how much calibration a target requires. The limit is robust: pairing keeps it exact for non-Gaussian shared content, the projection keeps its guarantee under anisotropic noise, and selective abstention cannot close the gap. On paired clinical and wearable sleep EEG, where differences between participants act as calibration noise, the formula predicts the device shift left in new participants, and more recordings per person soon stop helping. Together, these results tell whether a correction that falls short needs a better method, more recordings, or more participants.