Liquid Gated Attention
2026-08-31 • Machine Learning
Machine Learning
AI summaryⓘ
The authors address the problem of analyzing time series data that comes at irregular intervals and over long periods. They introduce Liquid Gated Attention (LGA), a new method that processes these time intervals without needing slow step-by-step calculations, allowing fast and parallel computations. Using this, they build LFormer, a flexible model that learns from continuous-time data efficiently. Their approach works well on various tasks and datasets, handling noise and sparse data while being competitive with existing methods.
time seriescontinuous-time modelingparallel computationgating mechanismassociative memorytemporal dynamicslong-range dependenciessequence modelinglinear computational complexity
Authors
Yiheng Jiang, Yuanbo Xu, Yongjian Yang
Abstract
Real-world time series often exhibit irregular sampling and extended temporal horizons, requiring models to capture continuous-time dynamics across arbitrary intervals without prohibitive scaling costs. Discrete-time methods collapse variable time intervals into static positional steps; solver-dependent continuous-time models preserve temporal structure but rely on sequential integration, precluding parallelization; and solver-free approximations avoid this cost yet none couples observed time intervals with input-driven state modulation. We propose Liquid Gated Attention (LGA), a solver-free parallel temporal operator. By parameterizing an input-driven gating mechanism with observed time intervals, LGA introduces a continuous-time inductive bias and formulates hidden state evolution as a fast-weight associative memory, enabling parallel computation across the temporal dimension. Using matrix associativity in non-causal encoding and a prefix scan in causal encoding, LGA attains linear temporal complexity in sequence length in both modes. A sequence-level normalization bounds cumulative temporal decay for stable long-horizon optimization. Building on LGA, we instantiate LFormer, a modular backbone for continuous-time representation learning. Across six tasks and sixteen datasets spanning up to 17,984 steps, LFormer demonstrates long-range dependency modeling, fine-grained state tracking, and trajectory reconstruction from sparse and noisy observations, while delivering competitive performance against state-of-the-art discrete-time and continuous-time baselines with linear scaling efficiency.