Counterfactual Transition Graphs: Evaluating Cross-Class Transition Quality
2026-08-24 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors suggest a new way to understand how time-series classifiers make decisions by looking at the relationships between different classes, not just individual examples. They create a network called a counterfactual transition graph (CGT) that shows how likely it is to switch from one class prediction to another with minimal changes. Testing this on a hand-movement task, they find that the graph reveals patterns that are different from what standard accuracy measurements show. Their approach works with any counterfactual explanation method and highlights differences between replacement-based and gradient-based methods in how they explore class boundaries.
counterfactual explanationstime-series classifierscounterfactual transition graphprototypeproximity aware retrievalconfusion matrixgradient-based methodsreplacement-based methodsdata manifoldclassifier boundaries
Authors
Syed Muhammad Hamza Zaidi, Szymon Bobek, Grzegorz J. Nalepa, Myra Spiliopoulou
Abstract
Counterfactual (CF) explanations for time-series classifiers are usually evaluated one example at a time: what minimal edit flips this single window's prediction? We argue that the more informative question for diagnostic interpretability is structural: how does the classifier connect its own classes to each other? We propose a counterfactual transition graph (CGT) in which each node is a class and each edge weight is the CF reliability of the transition from one prototype to another under a proximity aware retrieval sweep. On a six-class hand-movement task, we induce a CGT that reveals a non-trivial topology, which is not predicted by the binary confusion matrix: it shows that counterfactual reachability does not align with classifier accuracy and even runs counter to it (Spearman $ρ=-0.37$ over the 15 pairs), i.e. the boundaries the classifier separates most confidently are among those an in-distribution edit can least often cross. Our framework is method agnostic, i.e. any CF-explainers can be used. Presently, we use it to juxtapose replacement-based CFs with gradient-based CFs; gradient-based methods reach almost any class by stepping off the data manifold, while replacement-based methods stay on it and fail on precisely the rigid boundaries.