machine learning engineers in physics

Abstract: How many latent variables are needed to fit the cosmic microwave background (CMB) temperature and polarisation power spectra when a neural network is given no cosmological model, and to what extent do they map onto the standard cosmological parameters? We use multi-task learning with variational autoencoders trained on CLASS-generated spectra (TT, TE, EE) over $2\le\ell\le2500$. No cosmological model is built into the network, and the number of latent variables, $L$, is varied over $L=5$, $6$ and $7$. Their fits are compared to the Planck 2018 data using $\hatχ^2$, AIC, BIC and Bayesian evidence. Their correspondence with cosmological parameters is measured with normalised mutual information post-training. The loss weights are set within a single training run by uncertainty weighting, with the weight of the disentanglement term either fixed (semi-automated, SA) or learned (completely automated, CA). The reconstruction loss is split over multipole ranges in three configurations that separate the roles of temperature and polarisation and of the low- and high-$\ell$ ranges. In four of the six combinations of configuration and strategy, the fit improves from $L=5$ to $L=6$ and not beyond, and $L=6$ is the best, or close to the best, in five of the six cases. So, six latent variables are needed, as many as the free parameters of the standard, model-dependent approach. SA gives clearer disentanglement, while CA needs no a priori choice of the disentanglement weight and has the higher evidence at $L=5$ and $6$: the two strategies are points on a trade-off between interpretability and fit to the data. Notably, the latent variables for $τ$ and $A_s$ share almost no information in any model, consistent with large-scale polarisation separating the two, as in standard analyses. A deep-learning approach can thus give a compact, data-driven parametrisation of the CMB spectra.

Abstract: We study the training dynamics of a feed-forward neural network (NN) through the analysis of the neural tangent kernel (NTK) in a phenomenologically relevant fit of parton distribution functions (PDFs) from deep inelastic scattering (DIS) data. The exact integration of the gradient flow derived in a previous single-flavour analysis is generalised to the multi-flavour case, yielding an analytical perspective on the relationship between data and methodology during training. The Shapley value recently used in the context of PDFs is derived within our statistical framework for observables that depend linearly on the PDFs, and we outline a geometric interpretation of training. We expose the role of gradient-based optimisation algorithms and of the parametrisation in the regularisation of the ill-posed inverse problem that affects the determination of PDFs from experimental data, demonstrating that the regularisation amounts to the incremental inclusion of NTK modes, each learned at the timescale set by the corresponding eigenvalue. We find that only a few leading modes of the NTK are necessary to reconstruct the target PDFs, while the noise of the data is carried by the sub-leading ones learned at later stages of training. Finally, we consider the impact of the initialisation on the training dynamics, and recover a transient behaviour observed in the previous analysis and in the machine learning (ML) literature. Beyond sharpening our understanding of the training dynamics, our framework can be applied to other parametrisations (such as fixed functional forms) and ML applications in physics, as well as to stress tests in the presence of inconsistent data.

Abstract: Reconstructing the mass of a heavy resonance from its decay products with missing energy is one of the central tasks that directly determine the sensitivity in new physics searches at collider experiments. Supervised learning approaches to this problem often struggle to generalize well due to the presence of various systematic uncertainties and distribution shifts. Exhausting all possible variations in the labeled data can be very compute-intensive, while a failure of the model to generalize can corrupt the reconstructed resonance widths that are critical in peak-hunting analyses. In this work, following the foundation model paradigm, we use a self-supervised approach to pre-train a transformer encoder with VICReg to learn an embedding invariant to various corruptions, then fine-tune it for mass regression on a heavy resonance with masses ranging from 2.5 to 6.5 TeV and a SUSY-like cascade decay into an eleven-body final state. We show that the pre-trained model reconstructs sharper resonance peaks and has a more stable performance under various realistic corruptions, compared to a supervised model of the same architecture trained on the same augmented data from scratch.

Abstract: Decays of beauty and charm hadrons provide sensitive probes of physics beyond the standard model, including decays with invisible particles, in which part of the final state leaves no reconstructed detector signature. The large heavy-flavour data samples recorded by the LHCb experiment at the CERN LHC, together with its precise tracking, displaced vertex reconstruction, and particle identification, make it particularly well suited to learning a map of reconstructed heavy-hadron decay environments directly from data. We propose to bring recent advances in jet flavour tagging at ATLAS and CMS to significantly improve on the performance of the current LHCb taggers and extend them to the reconstruction of heavy-flavour decays with several invisible particles in the final state. To achieve this, we introduce a self-supervised transformer architecture that learns the decay maps without flavour or exclusive-decay labels by inferring masked particle identification information and completing jets from which constituents have been removed. Across five classification tasks in simulated LHCb Open Data, the self-supervised model outperforms an otherwise identical transformer with random weights, and performs comparably to a fully supervised transformer. We achieve a tagging power of about 10\%. In addition, removing constituents from reconstructed exclusive decays also systematically increases the model anomaly score relative to random removals from the same heavy hadrons. We confirm this behaviour directly in 2017 LHCb proton-proton collision Open Data: the score increases for all eight studied heavy-flavour channels, and the signal region response exceeds that in the adjacent sidebands. These studies provide a proof of principle that mapping heavy-flavour decay environments through jets can transform flavour tagging in LHCb and extend the discovery reach for incomplete or otherwise unusual decays.