CoJEPA: Combining Contrastive Learning and JEPA for Global-Local Music Representations
2026-08-31 • Sound
SoundArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors studied two ways of teaching a model to understand music: one that predicts parts of a sequence (JEPA) and another that compares entire sequences (contrastive learning). They combined these methods into a new approach called CoJEPA, which improves learning by using the strengths of both without adding complexity to the model. This new method trains more stable and detailed music representations, especially for recognizing tones and harmonies. Their results showed CoJEPA works better or equally well on different music understanding tasks compared to using each method alone.
Joint-Embedding Predictive Architecture (JEPA)Contrastive LearningSelf-Supervised LearningMasked Sequence PredictionTeacher-Student ArchitectureExponential Moving Average (EMA)Latent SpaceRepresentation LearningMusic Information Retrieval (MIR)Inductive Bias
Authors
Gabriel Meseguer-Brocal, Yuexuan Kong, Romain Hennequin
Abstract
Joint-Embedding Predictive Architecture (JEPA) has shown strong performance in learning rich representations through self-supervised prediction in latent space. However, it typically relies on teacher--student architecture with an EMA to stabilise training, and can tend to yield uninformative representations. Contrastive learning is stable to train and produces strong global representations, but remains limited on local tasks by the global nature of its objective. In this work, we combine both into CoJEPA: a single shared backbone jointly trained with a JEPA objective on masked sequence tokens and a contrastive objective on the class token. The contrastive gradient provides stability, removing the need for an EMA teacher entirely, while JEPA enriches the sequence tokens via local predictions that contrastive learning alone cannot provide. Crucially, no extra parameters are added to the backbone: the same model is guided towards richer representations purely through the design of its training signal. CoJEPA takes the best of both worlds, outperforming or matching both individual methods across global and local MIR tasks, with a particularly strong advantage on tonal and harmonic understanding, and without any task-specific architectural changes. CoJEPA shows that combining objectives with complementary inductive biases can substitute for scale, encouraging future work to invest in smarter training objectives over ever-larger models.