Summary
Sometimes when researchers try to understand how AI language models work by intervening on parts of them, the model gives the right answer but by using unexpected hidden routes inside itself. The authors created secret pathways inside a popular AI model and tested hundreds of thousands of interventions to see which ones relied on these hidden paths. They found that existing ways to detect if an intervention uses these hidden routes often fail, but a new method called Hidden-Pathway Contribution can detect many cases when the hidden routes produce the answer. However, when the hidden routes behave like normal parts of the model, detecting them remains a big challenge.
What this means in practice
- •For neural network engineers: Identify when causal interventions on language models lead to correct outputs via unintended internal pathways to improve interpretability methods.
- •For ai safety teams: Detect pernicious divergence in mechanistic tools to ensure interventions reflect true model reasoning rather than artificial shortcuts.
Abstract
Causal interventions such as activation patching and distributed alignment search (DAS) are the main tool for making mechanistic claims about neural networks. Recent work showed that these interventions routinely push representations off the model's natural distribution, and that such divergence is sometimes harmless and sometimes pernicious: it can recruit pathways the model never uses on natural inputs, so that an intervention produces the expected answer through the wrong mechanism. No method currently tells the two cases apart. We make this question testable by planting hidden pathways inside pretrained language models; the pathways are silent on every benchmark prompt by construction, so which interventions depend on them is known exactly. Across 72 configurations and 100,800 interventions on GPT-2 small, we find three things. (i) Nearest-neighbour and local-PCA distances at the intervention site, as used in prior work, score below chance (AUROC 0.35-0.47) at picking out interventions that give the right answer through a planted pathway. (ii) Hidden-Pathway Contribution (HPC), a label-free test that clamps downstream units to the regime of natural runs with the same output and measures how much of the decision disappears, flags pathway-dominated interventions with AUROC >= 0.99 when the pathway shows up as unit-level out-of-regime activity, but fails when every unit stays within its natural range, which we identify as the open problem. (iii) Optimised interventions actively seek hidden pathways: on a gender task, DAS routes 90-95% of its successes through planted pathways for three of four families, and a downstream on-manifold penalty cuts this share to under 5% at a cost of 6-11 points of success rate. In unmodified GPT-2, successful interventions show almost no unit-level out-of-regime reliance.