Low-rank classifiers limit noise influence under adversarial attacks
Tail-Weight Control and Localized Generalization in Nearly Low-Rank Adversarial Classification
Machine LearningArtificial Intelligence
Summary
This paper looks at how machine learning classifiers that use only a few important features handle attacks that try to fool them by changing inputs slightly. The authors show mathematically and with experiments that good classifiers tend to ignore noisy features and focus on important ones. They prove that under specific conditions, these classifiers keep their accuracy even when bad data tries to trick them. Their work helps understand why and when classifiers resist certain attacks and how sample size or number of noisy features affects this.
What this means in practice
- •For machine learning engineers: Design classifiers that maintain accuracy by focusing on important features despite adversarial input modifications.
- •For data scientists in finance: Improve risk models by ensuring that predictors resist noise and adversarial manipulation while leveraging low-dimensional structures.
Tested on simulated data.
Authors
Kunyu Wang, Dehan Wang, Wenjun Chen
Abstract
We study norm-constrained linear classification under Eu clidean adversarial perturbations in a Gaussian model with a low-dimen sional informative subspace and an independent noise tail. For bounded ramp loss, we prove that a principal-space witness with risk below one half forces every near-optimal predictor to have small tail weight. A path-specific density bound yields constants without requiring positive tail variance. Under isotropic principal covariance, we establish a unique population minimizer and joint local growth. Boundary normalization then removes the common attack penalty from centered margins, giving localized finite-sample guarantees governed by principal dimension and total tail energy. Globalized growth removes the entrance condition at weaker constants; a model-aware comparison retains local guarantees. Experiments with twenty paired repetitions show decreasing excess risk and tail use with sample size, and nearly unchanged behavior when tail dimension grows at fixed total energy. Pure-noise controls and optimizer diagnostics clarify the scope and limitations of these conclusions.