Summary
Understanding how complex systems compute often depends on knowing their internal workings, not just what outputs they produce. The authors study when a specific measure of integrated information, which relates to how parts of a system causally interact, can be reliably inferred from limited and noisy experiments. They show that even systems with identical input-output behavior can have very different internal mechanisms, detectable with enough targeted trials. However, partial access to the system’s internal states can limit the precision of these inferences, sometimes significantly. Their work demonstrates that measuring such quantities precisely is possible but can be sensitive to subtle changes and requires careful experimental design.
What this means in practice
- •For ai system auditors: Quantify internal causal structure of AI components from limited noisy tests to better evaluate their processing mechanisms.
- •For neural hardware designers: Design experiments to certify complex causal interactions in hardware neural networks, ensuring desired information integration properties.
Abstract
Interpretability, AI auditing and assessments of machine consciousness increasingly rest on mechanism-level quantities: functionals of a system's causal transition law that describe how it computes, not what it outputs. Experiments see that law only through finite, partial and noisy interventions. We ask when such a quantity is identifiable from an experiment's access, continuous in the mechanism and certifiable from finite data, and answer exactly for a deliberately demanding case - the integrated information $\varphi$ of integrated information theory - in stochastic networks of four and five units with known ground truth. Behavior can be silent: a recurrent and a feed-forward network with identical input-output laws at their response delay carry 0.28 and 0 bits, yet internal recordings separate them within 4-53 trials. Partial access can be just as blind: in our examples, units left out of a one-step recording impose precision floors of 0.30-1.02 bits that no sample size lowers, and in all-but-self parity networks, recordings of any length see several hidden units only through the product of their fidelities - a symmetry that one intervention broke in exact checks. The measure has sharp edges: because $\varphi$ selects a partition, and the cause and effect it specifies, by one criterion and reports another, infinitesimal changes of mechanism that cross an edge change it by 25-100% at 99% of the edges we found in random networks, and about 1% of surrogate models that fit interventional data almost perfectly misreport it by more than 20%. One-sided claims survive: the same data certify a lower bound on both sides of an edge, and we prove $\varphi>1.004$ bits throughout a 64-dimensional region of mechanisms and certify it from about $10^7$ controlled transitions with false-certificate probability at most $α$. Measurability is thus a design question with computable answers.