Measuring Monosemanticity in Sparse Autoencoders via Latent Activation Coherence
2026-07-20 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors focus on explainable AI and specifically on understanding how simple neural networks called Sparse Autoencoders (SAEs) produce clear, single-meaning features. They created a new way to measure how well these features represent one concept without needing extra labeled data or complex models. Their new metric, called the Tversky Monosemanticity Score (TMS), works better than existing methods when the original neural network has an irregular structure. They tested TMS on different setups and found it aligns well with other known measures, while also giving useful insights into how SAEs learn.
Explainable Artificial IntelligenceSparse AutoencodersMonosemanticityTversky Monosemanticity ScoreEncoder AnisotropyPretrained Vision ModelsDINOv3CLIPBLIP2Feature Interpretation
Authors
Katarzyna Filus, Sebastian Pokuciński
Abstract
Within Explainable Artificial Intelligence, mechanistic interpretability uses Sparse Autoencoders (SAEs) to extract more interpretable features from neural representations. However, assessing their monosemanticity, and thus explanation quality, remains challenging. Existing metrics require external concept labels or depend on pretrained embedding models, making them sensitive to encoder's geometry. We introduce the Tversky Monosemanticity Score (TMS), a label-free metric that operationalizes monosemanticity as activation-set coherence of binarized SAE latents, and does not require external embedding encoders. We evaluate TMS on SAEs trained on features from pretrained vision and vision-language models (DINOv3, CLIP, BLIP2), two common SAE regimes (TopK, BatchTopK), multiple sparsity levels, and expansion factors. Our results show that TMS is less affected by encoder anisotropy than its embedding-based alternative, while remaining aligned with established monosemanticity indicators. TMS also reveals distinct SAE training dynamics across base models. Moreover, under encoder anisotropy, TMS provides a stronger indication of probe-based concept deletion effectiveness, while being competitive otherwise.