RL-FAT: Reinforcement Learning for Fair Adversarial Training
Machine Learning
Summary
The authors address a problem where some object classes in deep learning models become less protected against tricky attacks, even if the overall defense seems good. They propose a new training method called RL-FAT that borrows ideas from reinforcement learning to pay more attention to classes that are easily fooled. This method helps the model become more fair by balancing how well it defends each class against attacks. Their experiments show that RL-FAT keeps good defense performance while improving fairness across different classes.
Authors
Tejaswini Medi, Levan Mikeladze, Margret Keuper
Abstract
Deep neural networks remain highly vulnerable to adversarial perturbations, and adversarial training (AT) has become a widely used approach for improving robustness. However, improvements in average robust accuracy often mask substantial class-wise disparities: while some classes become more robust, others may remain disproportionately vulnerable under attack. This imbalance raises an important adversarial fairness concern, particularly in vision tasks where reliable robustness is expected across all categories. To address this challenge, we propose \textbf{RL-FAT}, a reinforcement-learning-inspired fair adversarial training framework that uses policy-gradient based feedback from adversarial predictions. RL-FAT interprets the prediction distribution as a policy and combines correctness-based prediction rewards with class-wise value estimates to compute class-specific advantages for policy-gradient optimization. This enables the model to adaptively focus on class-wise misclassification. Furthermore, we introduce a fairness-emphasis adversarial loss that assigns stronger training pressure to classes with high adversarial loss, thereby mitigating class-wise robustness disparity. By combining reinforcement-driven adaptation with fairness-emphasis regularization, RL-FAT improves adversarial robustness while promoting a more balanced robustness distribution across classes. Extensive experiments demonstrate that our method achieves competitive robust accuracy and substantially reduces class-wise robustness imbalance compared with standard adversarial training baselines.