Segment-Anything-Model-2: Optimized for Qualcomm Devices

SAM 2, the successor to Meta's Segment Anything Model (SAM), is a cutting-edge tool designed for comprehensive object segmentation in both images and videos. It excels in handling complex visual data through a unified, promptable model architecture that supports real-time processing and zero-shot generalization.

This is based on the implementation of Segment-Anything-Model-2 found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime	Precision	Chipset	SDK Versions	Download
ONNX	float	Universal	QAIRT 2.42, ONNX Runtime 1.24.1	Download
ONNX	w8a8	Universal	QAIRT 2.42, ONNX Runtime 1.24.1	Download
QNN_DLC	float	Universal	QAIRT 2.43	Download
QNN_DLC	w8a8	Universal	QAIRT 2.43	Download
TFLITE	w8a8	Universal	QAIRT 2.43, TFLite 2.17.0	Download

For more device-specific assets and performance metrics, visit Segment-Anything-Model-2 on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

Custom weights (e.g., fine-tuned checkpoints)
Custom input shapes
Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Segment-Anything-Model-2 on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

Model checkpoint: sam2.1_hiera_t
Input resolution: 720p (720x1280)
Number of parameters (SAM2Encoder): 33.5M
Model size (SAM2Encoder) (float): 128 MB
Number of parameters (SAM2Decoder): 6.22M
Model size (SAM2Decoder) (float): 23.7 MB

Performance Summary

Model	Runtime	Precision	Chipset	Inference Time (ms)	Peak Memory Range (MB)	Primary Compute Unit
SAM2Decoder	ONNX	float	Snapdragon® 8 Gen 3 Mobile	4.742 ms	16 - 332 MB	NPU
SAM2Decoder	ONNX	float	Qualcomm® QCS8550 (Proxy)	6.749 ms	16 - 23 MB	NPU
SAM2Decoder	ONNX	float	Snapdragon® 8 Elite For Galaxy Mobile	3.451 ms	1 - 232 MB	NPU
SAM2Decoder	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	3.16 ms	17 - 241 MB	NPU
SAM2Decoder	ONNX	float	Snapdragon® X2 Elite	3.242 ms	15 - 15 MB	NPU
SAM2Decoder	ONNX	w8a8	Snapdragon® X Elite	12.498 ms	4 - 4 MB	NPU
SAM2Decoder	ONNX	w8a8	Snapdragon® 8 Gen 3 Mobile	9.214 ms	4 - 276 MB	NPU
SAM2Decoder	ONNX	w8a8	Qualcomm® QCS6490	132.622 ms	47 - 57 MB	CPU
SAM2Decoder	ONNX	w8a8	Qualcomm® QCS8550 (Proxy)	12.448 ms	2 - 11 MB	NPU
SAM2Decoder	ONNX	w8a8	Qualcomm® QCM6690	58.086 ms	49 - 69 MB	CPU
SAM2Decoder	ONNX	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	6.697 ms	2 - 240 MB	NPU
SAM2Decoder	ONNX	w8a8	Snapdragon® 7 Gen 4 Mobile	51.096 ms	41 - 58 MB	CPU
SAM2Decoder	ONNX	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	5.27 ms	4 - 264 MB	NPU
SAM2Decoder	ONNX	w8a8	Snapdragon® X2 Elite	5.395 ms	4 - 4 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® X Elite	5.439 ms	16 - 16 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® 8 Gen 3 Mobile	3.556 ms	4 - 230 MB	NPU
SAM2Decoder	QNN_DLC	float	Qualcomm® QCS8550 (Proxy)	5.271 ms	16 - 18 MB	NPU
SAM2Decoder	QNN_DLC	float	Qualcomm® QCS8450 (Proxy)	8.877 ms	4 - 229 MB	NPU
SAM2Decoder	QNN_DLC	float	Qualcomm® SA8295P	7.958 ms	0 - 214 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® 8 Elite For Galaxy Mobile	2.923 ms	4 - 215 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® 8 Elite Gen 5 Mobile	2.538 ms	4 - 202 MB	NPU
SAM2Decoder	QNN_DLC	float	Snapdragon® X2 Elite	2.999 ms	16 - 16 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® X Elite	2.247 ms	4 - 4 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® 8 Gen 3 Mobile	1.288 ms	0 - 105 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® QCS8275 (Proxy)	3.877 ms	4 - 82 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® QCS8550 (Proxy)	1.905 ms	4 - 37 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® SA8775P	2.56 ms	4 - 85 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® QCM6690	9.313 ms	4 - 265 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® QCS8450 (Proxy)	2.723 ms	0 - 100 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® SA7255P	3.877 ms	4 - 82 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Qualcomm® SA8295P	2.96 ms	4 - 81 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	1.035 ms	1 - 83 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® 7 Gen 4 Mobile	2.585 ms	4 - 204 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	0.901 ms	1 - 105 MB	NPU
SAM2Decoder	QNN_DLC	w8a8	Snapdragon® X2 Elite	1.317 ms	4 - 4 MB	NPU
SAM2Decoder	TFLITE	w8a8	Snapdragon® 8 Gen 3 Mobile	9.268 ms	0 - 257 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® QCS8275 (Proxy)	20.931 ms	0 - 211 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® QCS8550 (Proxy)	12.615 ms	0 - 22 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® SA8775P	51.668 ms	0 - 213 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® QCM6690	50.592 ms	12 - 205 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® QCS8450 (Proxy)	15.831 ms	0 - 240 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® SA7255P	20.931 ms	0 - 211 MB	NPU
SAM2Decoder	TFLITE	w8a8	Qualcomm® SA8295P	14.202 ms	0 - 223 MB	NPU
SAM2Decoder	TFLITE	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	6.812 ms	0 - 234 MB	NPU
SAM2Decoder	TFLITE	w8a8	Snapdragon® 7 Gen 4 Mobile	16.551 ms	10 - 206 MB	NPU
SAM2Decoder	TFLITE	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	5.426 ms	0 - 239 MB	NPU
SAM2Encoder	ONNX	float	Snapdragon® 8 Gen 3 Mobile	144.522 ms	45 - 3849 MB	NPU
SAM2Encoder	ONNX	float	Qualcomm® QCS8550 (Proxy)	198.49 ms	0 - 88 MB	NPU
SAM2Encoder	ONNX	float	Snapdragon® 8 Elite For Galaxy Mobile	112.516 ms	33 - 1665 MB	NPU
SAM2Encoder	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	97.137 ms	45 - 2288 MB	NPU
SAM2Encoder	ONNX	float	Snapdragon® X2 Elite	99.796 ms	71 - 71 MB	NPU
SAM2Encoder	ONNX	w8a8	Snapdragon® X Elite	88.875 ms	47 - 47 MB	NPU
SAM2Encoder	ONNX	w8a8	Snapdragon® 8 Gen 3 Mobile	61.169 ms	7 - 1993 MB	NPU
SAM2Encoder	ONNX	w8a8	Qualcomm® QCS6490	2388.433 ms	767 - 789 MB	CPU
SAM2Encoder	ONNX	w8a8	Qualcomm® QCS8550 (Proxy)	88.623 ms	0 - 69 MB	NPU
SAM2Encoder	ONNX	w8a8	Qualcomm® QCM6690	1344.194 ms	662 - 676 MB	CPU
SAM2Encoder	ONNX	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	58.353 ms	10 - 1793 MB	NPU
SAM2Encoder	ONNX	w8a8	Snapdragon® 7 Gen 4 Mobile	1255.793 ms	658 - 672 MB	CPU
SAM2Encoder	ONNX	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	39.583 ms	13 - 1884 MB	NPU
SAM2Encoder	ONNX	w8a8	Snapdragon® X2 Elite	42.311 ms	51 - 51 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® X Elite	202.655 ms	12 - 12 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® 8 Gen 3 Mobile	145.347 ms	12 - 2000 MB	NPU
SAM2Encoder	QNN_DLC	float	Qualcomm® QCS8550 (Proxy)	204.489 ms	12 - 15 MB	NPU
SAM2Encoder	QNN_DLC	float	Qualcomm® QCS8450 (Proxy)	394.742 ms	12 - 1950 MB	NPU
SAM2Encoder	QNN_DLC	float	Qualcomm® SA8295P	303.689 ms	0 - 1619 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® 8 Elite For Galaxy Mobile	115.423 ms	12 - 1675 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® 8 Elite Gen 5 Mobile	95.461 ms	12 - 1652 MB	NPU
SAM2Encoder	QNN_DLC	float	Snapdragon® X2 Elite	98.218 ms	12 - 12 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® X Elite	85.347 ms	3 - 3 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® 8 Gen 3 Mobile	58.758 ms	3 - 2046 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® QCS8275 (Proxy)	149.579 ms	3 - 1666 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® QCS8550 (Proxy)	83.146 ms	3 - 6 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® SA8775P	85.704 ms	3 - 1666 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® QCM6690	1678.346 ms	3 - 1891 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® QCS8450 (Proxy)	144.778 ms	3 - 2019 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® SA7255P	149.579 ms	3 - 1666 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Qualcomm® SA8295P	95.073 ms	3 - 1670 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	43.012 ms	3 - 1652 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® 7 Gen 4 Mobile	113.562 ms	3 - 1790 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	80.046 ms	3 - 1689 MB	NPU
SAM2Encoder	QNN_DLC	w8a8	Snapdragon® X2 Elite	81.915 ms	3 - 3 MB	NPU
SAM2Encoder	TFLITE	w8a8	Snapdragon® 8 Gen 3 Mobile	61.739 ms	4 - 1983 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® QCS8275 (Proxy)	150.783 ms	4 - 1741 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® QCS8550 (Proxy)	89.601 ms	4 - 7 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® SA8775P	88.434 ms	4 - 1740 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® QCM6690	2458.538 ms	6 - 257 MB	CPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® QCS8450 (Proxy)	149.431 ms	4 - 2044 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® SA7255P	150.783 ms	4 - 1741 MB	NPU
SAM2Encoder	TFLITE	w8a8	Qualcomm® SA8295P	101.316 ms	4 - 1758 MB	NPU
SAM2Encoder	TFLITE	w8a8	Snapdragon® 8 Elite For Galaxy Mobile	56.9 ms	2 - 1783 MB	NPU
SAM2Encoder	TFLITE	w8a8	Snapdragon® 7 Gen 4 Mobile	1278.97 ms	50 - 317 MB	CPU
SAM2Encoder	TFLITE	w8a8	Snapdragon® 8 Elite Gen 5 Mobile	39.288 ms	4 - 1861 MB	NPU

License

The license for the original implementation of Segment-Anything-Model-2 can be found here.

References

Community

Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
For questions or feedback please reach out to us.

Downloads last month: -; Downloads are not tracked for this model. How to track

Paper for qualcomm/Segment-Anything-Model-2

SAM 2: Segment Anything in Images and Videos

Paper • 2408.00714 • Published Aug 1, 2024 • 120