Adversarial training improves model confidence with geometric attacks
Structured Confidence Perturbation via Fisher Geometry for Adversarial Training
Machine LearningComputer Vision and Pattern Recognition
Summary
Machine learning models can be tricked by special changes called adversarial attacks that make the model more wrong or confused. However, some attacks only change the model's error number without truly confusing its confidence in predictions. The authors propose new ways to create attacks that better disturb the model's confidence patterns and to defend against these attacks. They use math ideas from geometry and statistics to find smarter directions to change inputs, making the model more robust to attacks. Their experiments show these methods help models resist tricky inputs better than previous approaches.
What this means in practice
- •For machine learning engineers: Enhance the robustness of AI models deployed in safety-critical applications by directly targeting confidence pattern changes during training.
- •For cybersecurity teams: Develop stronger defenses against adversarial input attacks by applying geometry-based perturbation methods that exploit model confidence properties.
Authors
Chenlong Wei, Hanling Wang, Chong Zhang, Qinkai Yu, Fangyu Wu, Xiaobo Jin
Abstract
Adversarial training is often plagued by "pseudo-attacks", perturbations that significantly increase the loss value but fail to truly alter the model's confidence structure. To address this, we propose two complementary approaches to directly attack and defend against confidence structure deformation. First, building upon the traditional loss maximization perspective, we introduce a hybrid objective that integrates cross-entropy and KL divergence, encouraging adversarial examples to simultaneously disrupt prediction accuracy and distribution consistency. Second, we geometrically reconstruct adversarial training by generating perturbations that maximize the distribution distance on the confidence manifold. Theoretical analysis shows that the optimal perturbation direction aligns with the principal eigenvectors of the Fisher information matrix. We also propose an efficient algorithm to avoid high-dimensional matrix computations. Experiments on multiple datasets and model architectures demonstrate that our method outperforms existing methods in adversarial robustness and robust transferability, particularly excelling in structural perturbations within high-confidence regions.