SAM 3: Segment Anything with Concepts: Unified Model for Concept-Prompted Segmentation, Doubling Accuracy
Submitted in November 2025. SAM 3 is a unified model that detects, segments, and tracks objects in images and videos based on concept prompts (short noun phrases, image examples, or a combination). It constructs a high-quality dataset containing 4 million unique concept labels, including hard negative samples. The model consists of an image-level detector and a memory-based video tracker sharing a common backbone, decoupling recognition and localization via an existence head. It doubles accuracy on image and video concept segmentation tasks and improves SAM's capabilities in visual segmentation.
SAM 3 elevates segmentation from category-agnostic to concept-driven, achieving precise segmentation and tracking of arbitrary noun phrases or example images through a large-scale concept dataset and existence head design. This fills SAM's gap in semantic understanding, bringing segmentation models closer to human visual cognition, with significant value for autonomous driving, robotics, video editing, and other scenarios.
SAM 3 adopts an architecture where an image-level detector and a memory-based video tracker share a single backbone. Recognition and localization are decoupled via an existence head, improving detection accuracy. The data engine constructs a dataset with 4 million unique concept labels, including hard negative samples, covering images and videos. The model supports short noun phrases, image examples, or a combination as prompts, outputting segmentation masks and unique identities. It achieves twice the accuracy of existing systems on image and video concept segmentation.
SAM 3 elevates segmentation from pixel-level to semantic level, with direct application value for autonomous driving (identifying specific vehicles), robotics (grasping designated objects), video editing (tracking particular characters), and more. Its concept prompting capability can reduce annotation costs and promote the deployment of visual AI in more vertical industries.
Visual AI companies can evaluate integrating SAM 3 into image/video editing, autonomous driving perception, and robotic vision systems. It is recommended to first leverage its concept prompting capability to build customized segmentation applications, such as e-commerce product recognition and security target tracking. Open-source release lowers the adoption barrier.
Attention should be paid to SAM 3's generalization ability on open-world concepts and the long-term stability of video tracking. The coverage and bias of its 4 million concept labels need evaluation. Additionally, the model's deployment efficiency on mobile and edge devices will affect practical applications.