Weighted Score-Oriented Losses for Temporally Localized Event Prediction
2026-06-22 • Machine Learning
Machine Learning
AI summaryⓘ
The authors explain that simply checking if a prediction is right or wrong at each point in time is not enough for systems that detect events like anomalies or warnings. They propose a new way to train neural networks using a special loss function that focuses more on when alarms happen relative to the actual events, giving less penalty if an alarm happens just before an event. This method, called weighted score-oriented loss (wSOL), helps models do better at predicting events in time-sensitive situations. They tested wSOL against other methods and found it improves performance when timing really matters.
anomaly detectionchangepoint detectionneural networksloss functioncross-entropyweighted score-oriented lossconfusion matrixback-propagationtime-series predictionevent detection
Authors
Edoardo Legnaro, Sabrina Guastavino, Francesco Marchetti
Abstract
Operational event-detection systems are rarely assessed by pointwise accuracy alone. In anomaly detection, changepoint detection, and warning systems, the utility of an alarm depends on its temporal position relative to an event. This produces a score-loss mismatch. Neural networks are commonly trained with classical loss functions, such as cross-entropy, whereas deployment decisions are obtained by thresholding network predictions, merging alarms through post-processing rules, and evaluating them with event-based metrics defined by detection windows and false-alarm costs. This paper studies a temporally localized specialization of weighted score-oriented loss (wSOL) for event prediction. Starting from score-oriented losses based on expected confusion matrices and from the weighted SOL framework of Marchetti et al., we consider temporal weights that discount near-event false positives and reduce false-negative penalties when an event is preceded by an admissible alarm. The resulting objective is differentiable with respect to the network predictions, and therefore can be optimized by back-propagation. It can be instantiated with balanced accuracy, true skill statistic, F1, critical success index, and related confusion-matrix scores. We evaluate the proposed approach by comparing cross-entropy, unweighted score-oriented loss, and wSOL on three benchmark datasets for time-series event prediction and detection. The results show that wSOL can improve performance when the evaluation utility is localized in time and is not already encoded by the pointwise labels.