Proper loss functions improve competing risks models more than architecture
The Loss or the Architecture? What Matters in Survival Analysis with Competing Risks
Machine Learning
Summary
In settings where different risks can cause an event, like health problems or machine failures, it's important to predict which risk will happen first. The authors studied how much deep learning model design and the way the model is trained each matter for these predictions. They found that the choice of loss function, which guides training, matters more than the model type. Using a specific loss that models risk probabilities directly gave much better results on many datasets. The model architecture still matters, especially matching it to the data type, but the training loss is the key factor for these problems.
What this means in practice
- •For hospital data teams: Improve models predicting which health complication will occur first by choosing the right loss function rather than focusing only on model type.
- •For industrial maintenance teams: Build better failure risk prediction tools by focusing on training losses that directly estimate competing failure probabilities.
Authors
Julie Alberge, Gaël Varoquaux, Judith Abécassis
Abstract
Often, \emph{eg} in healthcare and predictive maintenance, what matters is which of several competing causes will strike first, and with what probability: survival analysis with competing risks. Corresponding deep learning methods are developed as bundles: a new architecture together with a new training loss, tested on one to five datasets against a handful of baselines. Thus, a state-of-the-art result cannot be attributed to the architecture, the loss, or the evaluation protocol. The coupling is not accidental: with one incidence curve per cause, a network's head is tied to the losses expressible on it. We vary the two axes independently, three losses, two of them strictly proper, on four backbones, across two modalities that tease the axes apart: 20 tabular and 3 pathology-image datasets. One loss wins in both: modelling cumulative incidence directly under censoring weighting. It brings twice as much as benefits a good choice of backbone on competing-risks data, and no more than it on single-event data: the loss is decisive for competing risks. The backbone matters too, and must match the modality: tabular networks lead on tables, pretrained vision encoders on images. The two choices compound rather than substitute. Proper scoring rules reveal these dynamics; the concordance index (C-index), which is not proper, does not separate models: neither axis moves it, not even tuning on the loss built to improve it. This is an avenue of progress: composed, the two give a model nobody has tried, RealMLP or TabM trained with the cumulative-incidence loss, which takes the best Brier score in our benchmark.