Structured sparse autoencoders improve interpretation of time-series forecasts
TSAE: Structured Sparse Autoencoders for Interpreting Time-Series Forecasting Models
Machine LearningArtificial Intelligence
Summary
Time-series forecasting helps make important decisions like managing energy and monitoring the environment, but it is hard to understand what patterns the forecasting models rely on. The authors introduce TSAE, a new type of autoencoder that breaks down and organizes the internal patterns used by these models so they are easier to interpret. TSAE groups features by how variables relate and how they change over time, making it possible to analyze when and how strongly certain patterns appear. It also keeps the forecast results stable while providing clearer insights into the model’s internal workings.
What this means in practice
- •For energy system operators: Analyze forecasting models to improve reliability and detect failures in energy dispatch decisions using interpretable internal patterns.
- •For industrial control engineers: Use interpretable time-series representations to monitor and understand machine operation trends for better diagnostics and maintenance.
Authors
Baoxi Liu, Yi Xie
Abstract
Time-series forecasting informs critical decisions in energy dispatch, industrial operations, and environmental monitoring; understanding the patterns models rely on is essential for assessing reliability and identifying failures. Input attribution identifies important variables and time segments but offers limited insight into internal features, while standard sparse autoencoder (SAE) objectives do not directly constrain cross-variable structure or temporal continuity. We introduce TSAE, a structured sparse autoencoder for forecasting representations that decomposes hidden states into individually inspectable features. TSAE organizes cross-variable structure through shared and variable-routed private dictionaries, separates feature detection from magnitude estimation with gated encoding, and constrains neighboring sparse-code changes according to raw-segment similarity. These mechanisms support analysis of variable context, activation strength, and temporal evolution. Forecast-consistency fine-tuning further improves preservation of the frozen forecaster's outputs. The accompanying TSEVAL protocol separately audits fidelity, feature coherence, and physical calibration to ground feature interpretation. In three-seed experiments with frozen PatchTST on ETTh1, ETTh2, and ETTm1, TSAE achieves the lowest hidden-state reconstruction error, normalized forecast-reconstruction error (NFRE), and feature transition rate among five SAEs at comparable per-token activity. NFRE decreases by 4.3-26.4% relative to the next-best mean. Dataset-dependent tradeoffs in selectivity, physical correlation, and calibration show that fidelity and temporal-stability gains require independent semantic validation to support feature interpretation.