Inside-Out: Measuring Generalization in Vision Transformers Through Inner Workings
2026-04-09 • Machine Learning
Machine LearningComputer Vision and Pattern Recognition
AI summaryⓘ
The authors focus on how to tell if a machine learning model will work well on new, unseen data, especially when labeled examples are not available. They look at two main problems: choosing the best model before using it on new data, and checking if the model still works well after deployment when conditions change. Instead of just looking at the model’s outputs, the authors study the internal parts of the model, called circuits, to create new ways to predict performance. They propose two new measures that better estimate how well models generalize, showing improvements over older methods in different tests.
generalizationdistribution shiftmodel evaluationproxy metricscircuit discoveryinternal representationsdependency depth biascircuit shift scoremachine learning deploymentcausal interactions
Authors
Yunxiang Peng, Mengmeng Ma, Ziyu Yao, Xi Peng
Abstract
Reliable generalization metrics are fundamental to the evaluation of machine learning models. Especially in high-stakes applications where labeled target data are scarce, evaluation of models' generalization performance under distribution shift is a pressing need. We focus on two practical scenarios: (1) Before deployment, how to select the best model for unlabeled target data? (2) After deployment, how to monitor model performance under distribution shift? The central need in both cases is a reliable and label-free proxy metric. Yet existing proxy metrics, such as model confidence or accuracy-on-the-line, are often unreliable as they only assess model output while ignoring the internal mechanisms that produce them. We address this limitation by introducing a new perspective: using the inner workings of a model, i.e., circuits, as a predictive metric of generalization performance. Leveraging circuit discovery, we extract the causal interactions between internal representations as a circuit, from which we derive two metrics tailored to the two practical scenarios. (1) Before deployment, we introduce Dependency Depth Bias, which measures different models' generalization capability on target data. (2) After deployment, we propose Circuit Shift Score, which predicts a model's generalization under different distribution shifts. Across various tasks, both metrics demonstrate significantly improved correlation with generalization performance, outperforming existing proxies by an average of 13.4\% and 34.1\%, respectively. Our code is available at https://github.com/deep-real/GenCircuit.