Neuron activations enable efficient logical explanations for deep networks
Neuron Activation-based Computation of Logical Explanations for Deep Neural Networks
Logic in Computer ScienceMachine Learning
Summary
Understanding why deep neural networks make certain decisions is difficult but important. This paper presents a new way to explain these decisions by looking at the activations inside the network’s neurons rather than just input features. The authors use logical reasoning tools to create explanations that are flexible, scalable to deep networks, and come with strong guarantees. They tested their method on tasks like image recognition and medical data, showing it works efficiently even on complex models where earlier techniques struggle.
What this means in practice
- •For medical data teams: Generate reliable explanations of deep learning diagnoses for better trust and oversight in healthcare systems.
- •For image recognition developers: Provide scalable, logic-based explanations for deep models used in visual classification tasks to improve model transparency.
Authors
Tomáš Kolárik, Faezeh Labbaf, Fabrizio Leopardi, Grigory Fedyukovich, Michael Wand, Natasha Sharygina
Abstract
Formal explainability of classifying neural networks (NNs) is an active area of research, providing explanations with provable guarantees of the classification within continuous regions of the input feature space. However, the existing techniques are either limited to individual input features without guarantees on their relations or the provided solutions fail to scale to deep architectures. This paper addresses these issues by introducing a flexible symbolic framework for an efficient, guided computation of explanations of the NN behavior, parametrized by the activations of internal neurons, and using logical engines such as SMT solvers. Unlike prior methods that rely on specialized NN verifiers, our method yields explanations that are not restricted in shape. Our algorithm is implementable on top of a general-purpose logical solver, isolating the NN-specific encoding from the algorithmic framework. We experimented with a wide range of benchmarks from the domains of image recognition and medicine, illustrating the advantages of the new method, particularly in computational efficiency. Notably, our approach enables logical explanation of deep networks not amenable to prior logic-based methods.