Time series forecasting improves by learning channel sharing and specialization
Learning What to Share: Expert Specialization for Time-Series Forecasting
Machine LearningArtificial Intelligence
Summary
Forecasting many related time series together is hard because it's unclear when to share information across them or treat each one separately. The authors studied models that split the work between expert parts and found that linking specific data channels to fixed experts helps more than random or input-based assignments. They created a synthetic dataset to understand when specializing experts benefits forecasting and introduced a new method that learns the best way to share and specialize with fewer parameters. Their approach improved accuracy across multiple datasets while keeping most of the forecasting model fixed.
What this means in practice
- •For data scientists in finance: Improve forecasting of multiple financial indicators by assigning specialized prediction experts to different data channels, enhancing accuracy with fewer trainable parameters.
- •For energy demand planners: Enhance electricity demand forecasts across regions by learning which data streams benefit from shared or specialized prediction models, reducing error with efficient model updates.
Authors
Bashar Khoury, Shadi Omari, Assaf Schuster
Abstract
A central challenge in multivariate forecasting is deciding what to share across channels and where to specialize. Mixture-of-experts models offer flexible sharing, yet it remains unclear how much of their forecasting gains comes from added capacity or expert allocation. Through controlled comparisons across eleven datasets and three backbones, we show that informative channel assignments improve forecasting beyond matched shared capacity and random allocation. On frozen transformer backbones, persistent channel-specific mixtures outperform input-dependent routing on average. To examine when and how much specialization helps, we introduce GalaxyTS, a controlled synthetic dataset with exact shared and channel-specific forecasting references. It reveals how persistent channel mixtures exploit predictive differences concealed by short input windows and measures gains over matched shared corrections. These insights motivate Persistent Closed-Form Routing (PCR), a plug-and-play method that alternates exact conditional estimation of channel-specific mixture weights with gradient-based training of low-rank residual experts around a frozen pretrained forecaster. Averaged over four forecasting horizons, PCR-K4 achieves dataset-balanced mean squared error reductions of 3.86-14.25% relative to the three frozen backbones. In matched comparisons, PCR outperforms competing specialization methods on average while requiring substantially fewer trainable parameters.