State space models improve time series analysis with log-signature compression
LogSig-SSM: Time-Series Modelling with Multi-Scale Log-Signature Compression for State-Space Models
Machine LearningArtificial Intelligence
Summary
Time-series data that is collected irregularly and at high frequencies is hard to analyze especially over long periods. The authors propose LogSig-SSM, which first compresses complex time series into shorter summaries using a mathematical tool called log-signatures, then uses a specialized model to understand long-term patterns efficiently. This approach works well even when data times are uneven and is faster and less resource-intensive than some existing methods. It was tested successfully on several datasets including physiological signals, weather data, and clinical records.
What this means in practice
- •For hospital data teams: Improve prediction of patient outcomes from irregular and high-frequency clinical measurements using efficient long-sequence models.
- •For weather forecasters: Enhance multivariate weather prediction accuracy on long and irregularly sampled datasets by compressing data into informative tokens.
Authors
Felix Oury, Nicolas Calvo Peiro, Reiko J. Tanaka
Abstract
Time-series data are often sampled irregularly at high frequencies and exhibit long-range dependencies, which makes long-horizon modelling difficult. Continuous-time models such as neural controlled differential equations (NCDEs) and neural rough differential equations (NRDEs) can handle irregular sampling, but they scale poorly to long sequences. Selective state-space models (SSMs) such as Mamba scale linearly with sequence length, but they provide limited recurrent mixing across hidden dimensions within a single block. We propose LogSig-SSM (Log-Signature Compression for State-Space Models), which first compresses long multivariate time series into a shorter sequence of tokens using multi-scale windowed log-signatures, and then processes these tokens with a selective SSM backbone. LogSig-SSM is scalable and robust to irregular sampling, combining log-signature tokens that capture higher-order cross-channel interactions with a selective SSM that models long-range dependencies. The model also admits a continuous-time interpretation as an NCDE/NRDE-style system driven by a log-signature-based input, in which selectivity induces an input-dependent rescaling of the latent dynamics. Across four benchmarks, namely long-sequence classification on UEA, high-frequency physiological regression on PPG-DaLiA, multivariate weather forecasting, and irregularly sampled clinical prediction on PhysioNet Sepsis, LogSig-SSM outperforms or matches strong SSM and continuous-time baselines while training up to $30\times$ faster and using up to $37\times$ less GPU memory than Mamba on the longest sequences.