Entropy measures reveal patterns in time series from brain and heart data
Revisiting entropy and entropy-related magnitudes of a symbolic sequence and their estimation
Information Theory
Summary
Measuring the randomness or complexity in sequences of data helps us understand underlying patterns. The authors revisit mathematical ways to quantify such complexity and connect them to known theories about information and algorithms. They show how to estimate these measures from real data using a popular compression method. Their open-source software makes this efficient. They demonstrate the approach on brain signals, oscillator networks, and heart rhythm data to find meaningful insights.
What this means in practice
- •For medical data analysts: Characterize and classify heart rhythm irregularities using entropy measures on electrocardiogram data to support diagnosis of atrial fibrillation.
- •For neuroimaging teams: Extract information about functional brain connectivity from fMRI time series by applying entropy-based complexity measures.
Authors
Efren Aragon-Perez, Ania Mesa-Rodriguez, Karel Garcia-Medina, Daniel Estevez-Moya, Amanda Mollineda-Rodriguez, Ernesto Estevez-Rams
Abstract
This article revisits entropy and related magnitudes (entropy density, effective measure complexity, and informational distance) for the analysis of sequential data. We present their mathematical foundations within information theory and ergodic theory, establishing connections between probabilistic Shannon entropy and algorithmic Kolmogorov complexity. A central result, Zivs theorem, demonstrates the asymptotic equivalence between the normalised number of distinct words and entropy density, providing the theoretical basis for estimation via Lempel-Ziv factorisation. We examine the convergence properties of these magnitudes and introduce estimation procedures for finite sequences, including a method for the effective measure complexity and the transfer entropy rate. An open-source software library implementing parallel algorithms for efficient Lempel-Ziv factorisation is presented. The practical utility of the framework is illustrated through three case studies: functional brain connectivity analysis from fMRI data, characterisation of spatiotemporal dynamics in coupled nonlinear oscillators, and electrocardiogram-based classification of atrial fibrillation, demonstrating how entropic measures capture meaningful structural and dynamical properties across diverse domains.