Local signals and confounds explain graph-based pairwise predictions better

Interpreting Supervised Pairwise Graph Representations: Local Evidence, Confounding, and Provable Post-hoc Correction

Machine Learning

Summary

This paper looks at how graphs that represent connections between pairs of items are used to predict things, like spotting errors. The authors found that just because a graph is accurate doesn't mean its overall shape or higher-order connections really matter. Sometimes, simple local information or factors like sequence length explain the results better. They also show a way to adjust predictions after the fact to avoid false signals. So, bigger patterns in these graphs should only be trusted after ruling out simpler explanations.

What this means in practice

  • For machine learning engineers: Improve model interpretation by checking for local evidence before trusting complex graph structures in supervised pairwise predictions.
  • For neuroimaging data analysts: Assess confounds like sequence length when using graph-based models to analyze fMRI data for emotional state decoding.

Authors

Ilya Brekin, Ariela Moldavskaya, Elina Sidorova, Ural Valeev

Abstract

Supervised pairwise graph representations convert pairwise predictive relations into sample-specific graphs, but graph-level accuracy does not by itself show that higher-order topology carries the signal. In a controlled Transformer error-detection task, supervised pairwise graphs outperform attention-as-adjacency (ROC-AUC $0.839$ versus $0.559$; the comparable no-confidence pair is $0.687$ versus $0.559$), but their apparent advantage over model confidence disappears after controlling for sequence length, which is almost perfectly recoverable from the pooled hidden state ($R^2=0.9995$). On published HCP Emotion ensemble graphs, signed node aggregation already acts as an ensemble of supervised pairwise predictions. Edge magnitude alone is near chance; topology-aware augmentations change ROC-AUC by at most $0.004$, with only sign-separated degree showing small positive changes across all tested graph sizes, while a validation-tuned three-layer GCN provides no measurable improvement over the signed node-summary baseline. A Fisher post-hoc corrector satisfies its distribution-agnostic guarantees across the evaluated feature representations and remains stable under reduced labelled budgets; in a near-error-free fMRI calibration regime the bound correctly becomes statistically uninformative. A separate cross-attention analysis identifies a redundant reversal circuit and downstream EOS-timing and copy failures. Higher-order topology should therefore be credited only after local supervised evidence and recoverable nuisance variables have been excluded.