Residual cancellation improves stability of vision transformer explanations
ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformers
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
Summary
Vision transformers are special computer models that look at images, but it has been hard to explain exactly what parts of an image the model uses to make decisions. The authors found that a specific internal model part called residual connections causes instability in these explanations. They created a new method called ResLRP that adjusts how explanations handle these residual parts, making the results clearer and more reliable. This method works well across many types of vision transformers and helps show which image parts matter most.
What this means in practice
- •For computer vision engineers: Generate more stable and trustworthy explanations for vision transformer models to improve model interpretability.
- •For machine learning platform developers: Integrate residual-aware attribution methods to enhance explanation tools for models involving multimodal vision and language tasks.
Authors
Jim Berend, Reduan Achtibat, Daniel Schäffer, Alexander Binder, Wojciech Samek, Sebastian Lapuschkin, Maximilian Dreyer
Abstract
Vision Transformers (ViTs) are central to most modern vision models, yet obtaining input attributions that are fine-grained, faithful, and stable remains challenging. Layer-wise Relevance Propagation (LRP) has been adapted to transformer attention, but in ViTs it often produces noisy, unfaithful explanations. We show that the missing ingredient is the treatment of residual connections: cancellation effects in residual pathways lead to attribution explosion. Moreover, we find that these cancellations are substantially stronger in ViTs than in language transformers. To address this issue, we introduce Residual-aware Layer-wise Relevance Propagation (ResLRP), a simple extension of LRP whose propagation rules explicitly account for cancellations in residual branches, are exactly conservative, and provably bound relevance explosion. Causal channel-wise interventions confirm that residual cancellation, not a generic regularization effect, drives the instability. ResLRP substantially improves attribution quality across faithfulness and localization, evaluated on ViT architectures spanning supervised, self-supervised, contrastive, hierarchical, and multimodal families, as well as on the ground-truth-controlled FunnyBirds benchmark. The largest gains arise in modern Vision Language Models (VLMs), with +27-29% localization and up to 3.4x faithfulness scores. Beyond benchmarks, ResLRP localizes Sparse Autoencoder (SAE) features in input space, and our residual amplification measure serves as an architecture-level diagnostic predicting where attribution degrades.