Higher order models keep winning without higher order information
Do Higher-Order Models Win for Higher-Order Reasons? Rethinking Performance Gains in Hypergraph Learning
Machine LearningSocial and Information Networks
Summary
Some computer models try to learn from complex connections involving many items at once, called higher-order information. This paper shows that these complex models often do better than simpler models, but much of their success still remains even when the complex connections are changed or removed. The researchers found that tweaking simpler models by making better use of simpler, pairwise connections can close much of the performance gap. This means that the extra complexity may not be the main reason these models perform better.
What this means in practice
- •For machine learning engineers: Design faster and simpler models by improving pairwise feature propagation without needing complex hypergraph structures.
- •For data scientists in social networks: Use enriched pairwise weightings to better analyze network data without relying on higher-order models.
Authors
Fanchen Bu, Fan Li, Geon Lee, Sunwoo Kim, Xiaoyang Wang, Renaud Lambiotte, Kijung Shin
Abstract
Higher-order models (e.g., hypergraph neural networks) often outperform lower-order baselines on hypergraph learning benchmarks, and their advantages are commonly attributed to their ability to exploit higher-order information. However, better performance alone does not establish this explanation. We therefore ask: Do higher-order models win for higher-order reasons? To investigate this question, we introduce a controlled performance-attribution framework that perturbs higher-order information while preserving the lower-order, i.e., pairwise, information. Across 25 commonly used hypergraph learning benchmarks spanning three tasks, we frequently observe an intriguing pattern: higher-order models originally outperform lower-order baselines, yet retain most of their advantage after perturbation. This suggests that much of the observed advantage remains achievable without the higher-order information. We then investigate potential lower-order explanations for these remaining gaps. We find that simple additions to a lower-order baseline, e.g., richer pairwise weighting, more steps of pairwise feature propagation, and normalization, reduce the remaining performance gaps, supporting lower-order explanations for part of the observed advantage. Our analysis calls for the hypergraph learning community to rethink performance attribution by distinguishing performance gains from their explanations, adopt stronger lower-order baselines, and use suitable benchmarks that better test the value of higher-order information.