Neural networks improved by attacking concept logic instead of outputs
Attack logics, not outputs: Concept-based Property Attacks for Shortcut Elimination in Neural Networks
Cryptography and SecurityMachine Learning
Summary
Neural networks for computer vision can be tricked by tiny changes to images that cause them to make wrong decisions. The authors suggest focusing on errors in how the networks understand important concepts, like recognizing a red octagonal shape as a stop sign, instead of just checking if the final answer is right or wrong. They create attacks that try to break these logical rules and show that training networks to defend against such attacks helps them become more reliable and less biased. Their method works on different types of data including images and code, offering a new way to make AI systems more trustworthy.
What this means in practice
- •For machine learning engineers: Train vision models to resist attacks that exploit flawed concept reasoning instead of just class label errors.
- •For software security teams: Improve robustness of code analysis models by defending against concept-level adversarial manipulations.
Authors
Raik Dankworth, Felix Mächtle, Thomas Eisenbarth, Gesina Schwalbe
Abstract
Deep neural networks (NNs) for computer vision are vulnerable to adversarial attacks, such that minuscule malicious changes to inputs may induce unintuitive outputs. One key approach to verify and mitigate such robustness issues is to falsify expected output behavior, which enables local robustness guarantees and adversarial training. Due to the black-box nature of NNs, current attacks only falsify a class of the final output, such as flipping from $\text{\texttt{stop_sign}}$ to $\neg \text{\texttt{stop_sign}}$. In this work we argue that robustness analysis should focus more generally on logical inconsistencies in concept-level reasoning rather than solely on output misclassification. We formalize this perspective through concept-based properties, i.e., constraints over human-interpretable concepts such as $\text{\texttt{red}} \land \text{\texttt{octogonal}} \implies \text{\texttt{stop_sign}}$ and study the falsification of such properties as a principled form of adversarial attack. Furthermore, an implementation of concept-based properties on already trained NNs is proposed using techniques from concept-based explainable artificial intelligence (C-XAI). For that we formally prove that attacks on concept-based properties are expected to have a reduced search space compared to simple class falsification, whilst arguably be more aligned with intuitive robustness targets. The practical effectiveness of our approach for logically informed adversarial training is shown in an empirical study across multiple models and datasets, including both vision and a security-relevant code processing task. Our results demonstrate that concept-level falsification generalizes classical adversarial attacks: Retraining on our novel attacks maintains robustness gains as compared to classical adversarial retraining whilst better eliminating semantic biases.