Forecast combinations can mislead unless selection removes dilution
Target alignment, dilution and forecast selection when cross-sectional forecasts share a common target
Machine Learning
Summary
When many forecasters predict the same event, their forecasts share a common underlying target. The authors show each forecast breaks down into parts aligned with this target and unrelated parts. This understanding reveals common ways to combine forecasts can mistakenly reward adding forecasters who just add noise, not useful information. They propose and test a careful method to select which forecasts to combine, showing how this improves results on practical financial ranking tasks, though no combination beat a no-information baseline in their tests.
What this means in practice
- •For financial analysts: Improve selecting among multiple rank forecasts when predicting asset performance to avoid combining unhelpful forecasts.
- •For machine learning engineers: Refine ensemble methods for models predicting ranked outcomes by accounting for shared forecast targets and noise dilution.
Tested on one dataset.
Authors
Masoud Soleimani
Abstract
Forecasters often score the same units per date against one standardized realized outcome. We show that every standardized forecast splits exactly into a component aligned with this common target and a component uncorrelated with it. Three consequences follow: forecast-error correlation largely mirrors forecast correlation and is therefore a poor measure of diversity; an equally weighted combination beats a no-information forecast only when average alignment is large relative to the combination's dispersion; and the gain from adding a forecaster separates into genuine improvement and mere dilution, which equal-weight admission can mistakenly reward. We develop a cautious selection rule, study it in simulations, and apply it to language-model forecasts of US equity rankings and mechanical signals ranking exchange-traded funds. Selection removes most dilution losses, but no combination beats the no-information forecast.