Overlaydx offers visual and numeric measure for forecasting accuracy
Overlay\_dx - Automating forecasting evaluation
Machine LearningArtificial Intelligence
Summary
Evaluating how well forecast models predict future data can be tricky because typical scores give numbers without clear meaning. The authors created overlay_dx, a method that shows visually how many predictions fall close to actual outcomes and also gives a combined score from this. This approach helps people see and compare model performance better, potentially improving forecasting decisions.
What this means in practice
- •For data science teams: Improve model selection by visually confirming prediction alignment with actual time series values and receiving a combined numeric score.
- •For financial analysts: Assess and compare the reliability of forecasting models with clearer visual and numerical feedback for market trend predictions.
Authors
Long Ngo, Mohammed Amine Chamli, Jonathan Rivalan, Thomas Jaillon
Abstract
Traditional evaluation metrics provides numerical values but often lack comprehensibility, hindering effective differentiation of model performances. Our work addresses this challenge by introducing overlay\_dx, a novel evaluation metric measuring the performance of time series prediction models. Overlay\_dx is a visual metric that represents the percentage of predictions falling within a confidence interval around actual values. Additionally, once evaluation results are plotted, overlay\_dx computes the area under the overlay curve, providing a quantitative measure of alignment between predicted and actual values across different thresholds and predictions. Through extensive experiments, we demonstrate that our approach offers a unified evaluation framework that combines both visual and numerical assessments, enabling improved model comparison and providing valuable insights for further research and optimization efforts in time series prediction.