Beat-Synchronous Tokenization for ECG Transformers
2026-08-31 • Machine Learning
Machine Learning
AI summaryⓘ
The authors looked at different ways to split ECG signals into pieces before feeding them into Transformer models. Instead of cutting signals into fixed time chunks, they tested splitting by individual heartbeats to keep important heartbeat features intact. They found that using beat-based tokenization worked as well or better than fixed chunks, while using fewer tokens and making the models more stable. This suggests that organizing ECG data by heartbeats is a smart and efficient alternative.
Transformer modelsECG (electrocardiogram)tokenizationbeat-synchronous tokenizationfixed temporal patchesR-R intervalAUROCAUPRCmasked pretrainingcontrastive pretraining
Authors
Ahmed Sameh, Nolan Wilson, Max Enderlein, Yogatheesan Varatharajah
Abstract
Transformer-based electrocardiogram (ECG) models commonly tokenize waveforms into fixed temporal patches. Though convenient, fixed patching can split heartbeat structures across token boundaries. We study beat-synchronous tokenization as a physiologically grounded alternative, comparing fixed patches with three beat-aligned strategies: resampled beats, adaptive pooled beats, and resampled beats augmented with R--R interval information. Experiments span two settings: 10-second 12-lead diagnostic classification on PTB-XL after MIMIC-IV-ECG masked pretraining, and 60-second single-lead rhythm classification on Icentia11k after patient-level contrastive pretraining. On PTB-XL, resampled beat tokens achieve the highest mean macro Area Under the ROC Curve (AUROC; 0.8945) and nearly match the best fixed-patch macro Area Under the Precision-Recall Curve (AUPRC; 0.7414), reducing average sequence length from 100 to 11.2 tokens. On Icentia11k, beat-synchronous tokenizers obtain comparable AUPRC to fixed patching with better stability across runs. These results suggest morphology-preserving beat tokenization is a compact, competitive alternative to fixed temporal patching.