MR-JEPA: A General Purpose Video Foundation Model for Cardiac MRI
MR-JEPA is a self-supervised video foundation model for cardiac MRI that extends LeJEPA to 3D spatiotemporal inputs using tubelet tokenization, spatiotemporal masking augmentation, and initialization from a 2D CMR foundation model. It is pretrained on multi-sequence data (cine, LGE, mapping) from 10,505 patients across two centers without annotations. Evaluated on six downstream tasks using a unified multi-view gated attention architecture, it outperforms compared methods on all five regression tasks, achieving LV EF MAE of 4.79% (r=0.764) and GLS MAE of 1.87 (r=0.805), with 21-27% MAE reductions over baselines on strain tasks.
MR-JEPA is a self-supervised video foundation model for cardiac MRI that extends LeJEPA to 3D spatiotemporal inputs through tubelet tokenization, spatiotemporal masking augmentation, and initialization from a 2D CMR foundation model. Unlike prior CMR video models limited to cine data, MR-JEPA is pretrained on multi-sequence data (cine, LGE, mapping) from 10,505 patients across two centers without annotations. The frozen encoder is evaluated on six downstream tasks using a unified multi-view gated attention architecture: LV ejection fraction, RV ejection fraction, three myocardial strains (GLS, GCS, GRS), and four-class disease detection. MR-JEPA outperforms other compared methods on all five regression tasks, including both a domain-specific CMR model pretrained on more data with text supervision and a natural-video foundation model, achieving an LV EF MAE of 4.79% (r=0.764) and a GLS MAE of 1.87 (r=0.805), with 21-27% MAE reductions over baselines on strain tasks.
MR-JEPA extends the LeJEPA architecture to 3D spatiotemporal inputs via tubelet tokenization and spatiotemporal masking augmentation, enabling self-supervised learning from multi-sequence cardiac MRI without annotations. Initialization from a 2D CMR foundation model likely provides a strong prior for 3D video representation. The unified multi-view gated attention architecture allows the frozen encoder to be applied across diverse downstream tasks, suggesting the learned representations capture general cardiac structure and function.
This work demonstrates that self-supervised video foundation models can be effectively pretrained on large-scale, multi-sequence medical imaging data without manual labels, potentially reducing annotation costs and enabling broader clinical applications. The model's superior performance over domain-specific and natural-video baselines indicates a shift toward general-purpose medical imaging foundation models that leverage spatiotemporal context.
MR-JEPA could reduce the need for expensive expert annotations in cardiac MRI analysis, enabling faster development of automated diagnostic tools. Its strong performance on ejection fraction and strain estimation suggests potential for clinical decision support, improving efficiency and consistency in cardiology. The model may be licensed or integrated into medical imaging software platforms.
Next observable signals include: (1) publication of peer-reviewed validation on external datasets; (2) release of model weights or code to facilitate adoption; (3) extension to other cardiac imaging modalities or diseases; (4) clinical deployment studies assessing impact on diagnostic accuracy and workflow; (5) comparisons with emerging video foundation models on additional tasks.