Fisher induced geometry improves understanding of model predictions
Fisher-IRG: Fisher-Induced Local Invariant Representation Geometry across Language and Vision Models
Machine LearningComputation and LanguageComputer Vision and Pattern Recognition
Summary
Some small changes in how machines see words or images can cause big differences in their answers, while other bigger changes might not matter much. The authors propose a new way, called Fisher-IRG, to measure which changes really affect the machine's understanding by looking at how sensitive the model's predictions are. They tested this method on language and image models and found it better at capturing meaningful changes than older methods. This helps us understand what parts of the machine’s internal view are important for meaning and which are just noise.
What this means in practice
- •For machine learning engineers: Identify and focus on meaningful internal variations in language and vision models to improve robustness and interpretability.
- •For ai system auditors: Pinpoint model behavior changes caused by semantically significant input shifts to detect vulnerabilities or biases.
Authors
Abdullah All Tanvir, Xin Zhong
Abstract
Semantic-preserving transformations can induce substantial motion in learned representations, while small changes may strongly affect model predictions, raising a basic question: what local metric best captures semantically consequential variation? We propose Fisher-induced invariant representation geometry (Fisher-IRG), which measures local representation directions through their predictive sensitivity. Around each representation, we construct semantic-preserving and semantic-changing neighborhoods, aggregate their local Fisher information, and recover invariant directions through a contrastive generalized eigenvalue problem. Controlled displacement analyses first show that comparable Euclidean motion can have substantially different predictive consequences, supporting the need for a predictive geometry. Across language and vision models, Fisher-IRG yields stronger semantic-versus-nuisance predictive selectivity and generally more reproducible subspaces than covariance-based geometry, while recovering systematically distinct local directions. Representation interventions further localize semantic effects to the Fisher-derived subspace, and held-out separation and retrieval show that the recovered geometry generalizes beyond the discovery neighborhoods. These results support Fisher-IRG as a principled framework for characterizing local invariant representation geometry.