Agreement among ai advisors signals risk more than forecast accuracy
When AI Advisors Agree: Distinguishing Forecast Risk from Forecast Reliability
Multiagent Systems
Summary
Sometimes when multiple AI tools agree on a prediction, people think it's more trustworthy. The authors looked at nearly 1,000 asset forecasts from several different AI methods and found that agreement often means the asset is just easier to predict, not that the forecasts are more accurate. They also found that while agreement helps identify if the price will move up or down, it doesn’t mean the predicted size of the change is more reliable. This shows that agreement between AI forecasts means different things and isn’t simply a sign of better prediction.
What this means in practice
- •For quantitative investment teams: Design forecast combining methods that treat agreement as evidence of asset risk rather than improved prediction precision.
- •For financial risk managers: Use forecast agreement patterns to better distinguish risky assets from truly reliable return predictions for risk assessment.
Authors
Russlan Ramdowar
Abstract
If seven AI advisors give similar forecasts, should we trust them more? Agreement can indicate useful shared evidence, but it can also reflect an asset that was easier to forecast in the first place. We examine this distinction using 928 completed first-quarter forecasts for 232 assets across four production batches of iPulse AI. Ten historical Gemini configurations are grouped into seven investment frameworks; their archived forecasts are compared with later market outcomes, without generating new reports. Mean absolute return error rises from 10.31 percentage points in the lowest-disagreement quartile to 21.05 in the highest. Yet a forecast of no price change shows almost the same increase, from 9.89 to 21.15. The relationship between disagreement and additional forecast error is not resolved after accounting for visible asset risk. Retaining the quarter of forecasts with lowest historical volatility produces lower error than retaining those with least disagreement: 7.97 versus 10.31 percentage points. Agreement about direction behaves differently. When at least six of seven frameworks share a direction, directional accuracy is higher, but return-magnitude error is also larger. These results distinguish three questions often collapsed into one confidence score: how risky the asset is, whether advisors agree on its direction, and how precisely they forecast its return. For this historical council, agreement is informative in specific ways; it is not a general certificate of forecast reliability.