Routing entropy offers limited new confidence on transformer predictions
Auditing Routing Entropy as an Uncertainty Signal in Attention-Residual Transformers
Artificial Intelligence
Summary
This paper studies whether a particular signal called routing entropy could tell if a transformer model's prediction is right or wrong beyond what the model's own confidence shows. The authors tested this on certain transformer models trained on image recognition tasks and found that routing entropy does not reliably improve prediction confidence. They checked this signal in multiple ways and found no consistent benefit beyond existing confidence measures. The findings suggest some small, controlled effects but no strong practical use so far.
What this means in practice
- •For machine learning developers: Understand that routing entropy signals in transformers provide little extra value for prediction reliability beyond confidence scores when calibrating models on image datasets.
- •For data engineers in ai systems: Design model monitoring pipelines that focus on output confidences rather than routing entropy signals for uncertainty estimation in transformer-based vision models.
Authors
Wenhao Liang, Lin Yue, Wei Emma Zhang, Mingyu Guo, Olaf Maennel, Weitong Chen
Abstract
Dynamic architectures leave a per-example routing trace beside each prediction, and diffuse routing is easy to read as a sign that the prediction is unreliable. We audit that reading for routing entropy in Attention-Residual (AR) variants of Swin-Tiny and DeiT-Small, trained from scratch on CIFAR-10/100 with a soft-binned calibration auxiliary loss, asking whether the trace carries information about correctness beyond what the model's own confidence already reveals. Three checks probe this increment: does a routing signal appear at fixed confidence, does it replicate across training seeds, and can a held-out predictor exploit it against output-only and shuffled-trace controls? A sensitivity audit then injects effects of known size and measures the fraction of each that the probes recover. No test in the fixed 30-test binned family survives multiplicity correction, and neither the nominal hit nor a borderline result recurs in its sibling seeds. Across 24 paired runs a scalar routing probe yields no pooled improvement in routing-stratified calibration, and an entropy-profile probe predicts correctness better than the same probe given shuffled profiles yet worse than a confidence-only predictor in both binary log-loss and Brier score: a gain over shuffled traces does not become a gain over the output. Conditioning on the complete logit vector leaves the corresponding comparison unresolved. The audit bounds how far these non-detections can be read: at an injected effect of 0.010 nats the profile probe recovers 24-59% of the oracle gain, and a reference-preserving correction probe recovers 8% and 23% in the two CIFAR-100 settings, below the threshold we fixed for applying it to real labels. The results establish control-dependent gains and incomplete estimator recovery, not the absence of conditional routing information.