ARIMA: Reconstruction-Grounded Predictive Representation Learning for Symbolic Music
2026-07-10 • Sound
Sound
AI summaryⓘ
The authors created ARIMA, a new method to understand symbolic music by looking at short chunks of music directly instead of focusing on individual music tokens. ARIMA learns to predict what comes next in music based on previous parts and reconstructs music details to keep important local information. Their tests show ARIMA works well for finding harmony, timing, and matching performances, sometimes better than larger models. They also found that predicting the next part is key for understanding music over time, and reconstructing music helps the learning process stay stable.
self-supervised learningsymbolic musiclatent representationcontrastive learningcausal predictionstructured reconstructionwindow-based encodingdownstream tasksmusic retrievaltemporal progression
Authors
Mingyang Yao, Zhaoxiang Feng
Abstract
Self-supervised learning for symbolic music has advanced largely through token-level pretraining, but such representations remain tied to tokenizer-specific sequences and often provide time-span-level embeddings only indirectly. In this paper, we propose ARIMA, a reconstruction-grounded latent predictive framework for symbolic music that learns compact window-based representations directly from data. ARIMA encodes each fixed-duration window into a continuous latent representation, trains a causal predictor with contrastive next-latent prediction, and grounds the encoder through structured reconstruction of music elements. This design preserves local musical details while modeling temporal progression across windows. We evaluate ARIMA on downstream tasks spanning various levels of music understanding. Results show that ARIMA is particularly efficient and effective on tasks involving harmonic, timing, and cross-performance retrieval, while remaining competitive with much larger baselines on other tasks. Ablations further show that next-latent prediction is essential for temporally integrated representations, and that structured reconstruction stabilizes latent learning without requiring explicit variance regularization. The code is at https://github.com/AndyWeasley2004/symbolic_music_wm.