Papers for

ai developers building graph-based tools

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Stable transformers improve deep graph generation performance

Stable Transformers for Graph Generation

Abstract: Graph generative models increasingly rely on Graph Transformers (GT) to capture complex dependencies among nodes and edges. While deeper architectures should provide greater expressive capacity and a broader receptive field, their effectiveness can decline with depth: repeated self-attention progressively contracts node representations, impeding information flow and gradient propagation. We analyse this phenomenon from a dynamical systems perspective, focusing on how the denoiser's spectral dynamics affect graph generation. We show that standard GT denoisers become increasingly dissipative as depth grows, leading to vanishing gradients and representation collapse. To isolate the effect of these dynamics, we construct a permutation-equivariant GT with inherently stable, non-dissipative transport. We also introduce a damping mechanism that continuously interpolates between non-dissipative and increasingly contractive regimes, enabling a direct assessment of how dissipation influences generation. Experiments on synthetic and molecular graph generation benchmarks show that the gap between these regimes widens with depth: non-dissipative dynamics preserve representation diversity and gradient flow, sustaining strong generative performance, whereas greater contraction progressively impairs it. These findings identify the denoiser's dynamical regime as a key design factor for deep graph generative models.

Wed 30 SeptMachine Learning
The gist
Graph Transformers are tools used to create complex graph structures, but when made deeper, they can lose effectiveness as their internal signals shrink and become harder to train. The authors studied this shrinking problem and found that it causes loss of information and weakens learning. They created a new transformer design that avoids this shrinking, preserving information flow and improving how well graphs are generated, especially in deeper models. Their experiments showed that this stability leads to better results on synthetic and molecular graph tasks.
Open → 2609.39739v1