Efficient transformer improves particle jet tagging at colliders
This is SParTA: A modular, efficient and scalable transformer architecture for jet-tagging
Machine Learning
Summary
This paper introduces a new kind of computer model called SParTA that helps identify particles produced in powerful collisions in particle accelerators. It works by focusing on the most important parts first, then efficiently adding less important details without slowing down too much. The authors show that SParTA performs as well as or better than existing models but uses fewer computing resources. This can help scientists analyze particle collision data more quickly and accurately.
What this means in practice
- •For particle physicists: Improve classification of particle jets in collider experiments using a more efficient and scalable transformer model, enabling faster analysis of collision data.
- •For machine learning engineers: Build more resource-efficient transformer models for processing large physical datasets by applying a hybrid attention mechanism separating core and soft inputs.
Authors
Rajneil Baruah, Subhadeep Mondal, Sunando Kumar Patra, Satyajit Roy
Abstract
We present the Sparse Particle-interaction Transformer Architecture (SParTA), a modular, scalable, and efficient transformer for jet tagging at high-energy colliders. Exploiting the steeply falling transverse-momentum ($p_T$) spectrum of jet constituents, SParTA factorizes constituent processing into two topological regimes. Ordered by descending $p_T$, only the $N$ hardest constituents (the hard core) participate in quadratic $O(N^2)$ differential interaction attention driven by pairwise physical and learned relational features. The remaining soft constituents are integrated via linear $O(lN)$ cross-attention to the hard core, preserving soft-radiation and color-flow information while avoiding the $O(l^2)$ memory bottleneck of full self-attention over all $l$ constituents. Pairwise interaction features are computed dynamically in-network from constituent four-momenta and conditioned on global jet-level substructure observables. On the quark-gluon tagging benchmark, a 300K-parameter SParTA model achieves an AUC of 92.02% and accuracy of 85.06%, matching the 2.14M-parameter Particle Transformer (ParT) at one-seventh the parameter count and outperforming IAFormer by 21% in background rejection at 30% signal efficiency. On the boosted top-tagging benchmark, SParTA outperforms ParT and ParticleNet with 511K parameters, while we analyze the architectural trade-offs relative to fully dense multi-layer interaction networks.