CoJEPA: Combining Contrastive Learning and JEPA for Global-Local Music Representations
CoJEPA combines contrastive learning and Joint-Embedding Predictive Architecture (JEPA) in a single shared backbone, jointly trained with a JEPA objective on masked sequence tokens and a contrastive objective on the class token. The contrastive gradient removes the need for an EMA teacher. No extra parameters are added to the backbone. CoJEPA outperforms or matches both individual methods across global and local music information retrieval (MIR) tasks, with a particularly strong advantage on tonal and harmonic tasks.
A research paper introduces CoJEPA, a method that combines contrastive learning and JEPA for music representation learning. It uses a single shared backbone trained with both a JEPA objective on masked sequence tokens and a contrastive objective on the class token. The contrastive gradient stabilizes training, eliminating the need for an EMA teacher. CoJEPA achieves better or comparable performance to individual methods on global and local MIR tasks, especially tonal and harmonic tasks.
CoJEPA demonstrates that combining contrastive and predictive objectives in a single backbone can yield richer representations without additional parameters. The contrastive objective on the class token provides stability, replacing the EMA teacher typically required in JEPA. The JEPA objective on masked sequence tokens enriches local representations, addressing a limitation of purely contrastive methods. This approach may be applicable to other domains where both global and local representations are needed.
Improved music representation learning can enhance music information retrieval tasks such as genre classification, key detection, and chord recognition. This could benefit music streaming services, recommendation systems, and audio analysis tools. The removal of the EMA teacher simplifies training and may reduce computational overhead, making the method more accessible for industry applications.
CoJEPA offers a more efficient training method for music representation models by eliminating the EMA teacher and achieving strong performance on both global and local tasks. This could reduce training costs and improve model quality for music-related products, potentially leading to better user experiences and new features in music platforms.
Future work may explore applying CoJEPA to other audio domains or modalities, and investigate scaling to larger models and datasets. The approach could inspire hybrid training objectives in other self-supervised learning contexts. Observing whether CoJEPA is adopted in downstream MIR benchmarks or integrated into production systems will be a key signal.