MedSAM2: Segment Anything in 3D Medical Images and Videos: Medical Image Segmentation Foundation Model Reduces Manual Annotation Cost by 85%
In April 2025, the research team released MedSAM2, a promptable medical image segmentation foundation model based on fine-tuning SAM2, supporting 3D images and videos. The model was trained on over 455,000 pairs of 3D image-mask and 76,000 frames of data, surpassing previous models across multiple organs, lesions, and imaging modalities. Through a human-in-the-loop pipeline, the team conducted the largest user study to date, including annotation of 5,000 CT lesions, 3,984 liver MRI lesions, and 251,550 echocardiogram video frames, demonstrating that MedSAM2 can reduce manual annotation costs by over 85%. The model has been integrated into commonly used platforms, supporting local and cloud deployment.
MedSAM2 extends SAM2's general segmentation capabilities to medical 3D and video domains, addressing the core pain point of high medical image annotation costs. The 85% reduction in annotation cost significantly lowers the barrier to building large-scale medical image datasets, accelerating the deployment of AI in precision medicine. The model's excellent performance across multiple modalities and tasks makes it an infrastructure-level tool for medical image analysis.
MedSAM2 is based on the SAM2 architecture, gaining domain adaptation through fine-tuning on large-scale medical datasets. Training data covers multiple modalities including CT, MRI, and ultrasound, with segmentation tasks such as organs, tumors, and lesions. The model supports various prompt types including points, boxes, and masks, enabling interactive segmentation. The human-in-the-loop pipeline adopts an iterative process of 'AI initial segmentation + manual correction', with efficiency gains validated in user studies. The model has been integrated into platforms such as 3D Slicer and MONAI, supporting local and cloud deployment. Evaluation metrics include Dice similarity coefficient and intersection over union, achieving state-of-the-art on multiple public and internal datasets.
This achievement has profound implications for the medical imaging industry. For hospitals and imaging centers, MedSAM2 can significantly reduce radiologists' annotation workload and improve diagnostic efficiency. For medical AI companies, the model serves as a foundational tool to accelerate new product development cycles. For medical device manufacturers, it can be integrated into PACS systems to provide real-time segmentation functionality. Additionally, the 85% cost reduction enables small and medium-sized hospitals to afford high-quality AI-assisted diagnosis.
It is recommended that medical AI companies and imaging device manufacturers immediately evaluate the integration potential of MedSAM2. A medical image annotation SaaS platform based on MedSAM2 can be developed, offering annotation services to hospitals and CROs. Investment directions include: collaborating with the team to develop specialized versions (e.g., ophthalmology, pathology), or building private deployment solutions based on the open-source model. Additionally, attention should be paid to the model's expanded applications in surgical navigation and radiotherapy planning.
Attention should be paid to MedSAM2's generalization ability on rare lesions and low-quality images, as well as the model's actual deployment effectiveness in clinical workflows. Key signals include: regulatory certification progress from agencies like the FDA, integration cases with mainstream PACS systems, and performance validation in real clinical environments. Furthermore, the model's continual learning capability (e.g., adapting to new modalities) is also critical for long-term application.