Multi-Scale Convolution with Optimal Transport Attention Effect on Multivariate Time Series
2026-07-12 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors address challenges in analyzing multivariate time series data by proposing MSC-OT, a method that combines multi-scale convolution with an optimal transport-based attention mechanism. They use inverted embedding to better capture relationships between variables and enhance attention with convolution and Sinkhorn optimal transport to balance information flow. Their experiments on several common datasets show improved forecasting accuracy, and tests removing parts of their method confirm that each component helps. Overall, their work improves how models understand complex time series with multiple variables.
Multivariate Time SeriesMulti-Scale ConvolutionOptimal TransportAttention MechanismInverted EmbeddingSinkhorn AlgorithmTime Series ForecastingAblation StudyCross-Variate RelationshipsMatrix Scaling
Authors
HaoChong Fu, Jian Xu
Abstract
The analysis of Multivariate Time Series (MTS) plays an important role in a lot of real-world practical applications, but it still remains some challenging problem about capturing multi-granularity structural patterns and suppressing noise appropriately. Multi-Scale Convolution with Optimal Transport Attention (MSC-OT) is proposed in this paper. MSC-OT is a useful architecture to optimize the attention mechanism. It combines multi-scale convolution with Sinkhorn optimal transport method based on inverted embedding. The inverted embedding approach embeds each variable as a token and allows the model to capture cross-variate relationships better. MSC-OT consists of two part: (1) Multi-Scale Convolution Enhancement, that applies multi-scale convolutions to attention score matrices based on inverted embedding, capturing local structural patterns in the variate-interaction space induced by compressed temporal representations; (2) Sinkhorn Optimal Transport Regularization, that formulates attention computation as an optimal transport problem and employs iterative matrix scaling to ensure balanced information flow across variates. Adaptive Fusion Strategy utilizes softmax-normalized learnable weights to dynamically combine base attention, convolution-enhanced, and OT-regularized scores. Experiments on widely-used datasets, including ETT, Electricity, Traffic, Solar-Energy, and Exchange-Rate, show that MSC-OT achieves well performance in both short-term and long-term forecasting tasks. Ablation experiments further validate the effectiveness of each proposed component and their synergistic contributions to improving prediction accuracy for multivariate time series forecasting.