Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ tags:
|
|
16 |
|
17 |
Generates high resolution images from text prompts using a latent diffusion model. This model uses CLIP ViT-L/14 as text encoder, U-Net based latent denoising, and VAE based decoder to generate the final image.
|
18 |
|
19 |
-
This model is an implementation of Stable-Diffusion-v2.1 found [here](
|
20 |
This repository provides scripts to run Stable-Diffusion-v2.1 on Qualcomm® devices.
|
21 |
More details on model performance across various devices, can be found
|
22 |
[here](https://aihub.qualcomm.com/models/stable_diffusion_v2_1_quantized).
|
@@ -32,16 +32,23 @@ More details on model performance across various devices, can be found
|
|
32 |
- VAE Decoder Number of parameters: 83M
|
33 |
- Model size: 1GB
|
34 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
35 |
|
36 |
|
37 |
|
38 |
-
| Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
|
39 |
-
| ---|---|---|---|---|---|---|---|
|
40 |
-
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Binary | 11.633 ms | 0 - 1 MB | INT8 | NPU | [TextEncoder_Quantized.bin](https://huggingface.co/qualcomm/Stable-Diffusion-v2.1/blob/main/TextEncoder_Quantized.bin)
|
41 |
-
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Binary | 217.134 ms | 0 - 2 MB | INT8 | NPU | [VAEDecoder_Quantized.bin](https://huggingface.co/qualcomm/Stable-Diffusion-v2.1/blob/main/VAEDecoder_Quantized.bin)
|
42 |
-
| Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Binary | 101.094 ms | 0 - 2 MB | INT8 | NPU | [UNet_Quantized.bin](https://huggingface.co/qualcomm/Stable-Diffusion-v2.1/blob/main/UNet_Quantized.bin)
|
43 |
-
|
44 |
-
|
45 |
|
46 |
## Installation
|
47 |
|
@@ -97,30 +104,34 @@ device. This script does the following:
|
|
97 |
```bash
|
98 |
python -m qai_hub_models.models.stable_diffusion_v2_1_quantized.export
|
99 |
```
|
100 |
-
|
101 |
```
|
102 |
-
|
103 |
-
|
104 |
-
|
105 |
-
|
106 |
-
|
107 |
-
|
108 |
-
|
109 |
-
|
110 |
-
|
111 |
-
|
112 |
-
|
113 |
-
|
114 |
-
|
115 |
-
|
116 |
-
|
117 |
-
|
118 |
-
|
119 |
-
|
120 |
-
|
121 |
-
|
122 |
-
|
123 |
-
|
|
|
|
|
|
|
|
|
|
|
124 |
```
|
125 |
|
126 |
|
@@ -231,15 +242,19 @@ provides instructions on how to use the `.so` shared library or `.bin` context b
|
|
231 |
Get more details on Stable-Diffusion-v2.1's performance across various devices [here](https://aihub.qualcomm.com/models/stable_diffusion_v2_1_quantized).
|
232 |
Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
|
233 |
|
|
|
234 |
## License
|
235 |
-
|
236 |
-
|
237 |
-
|
|
|
238 |
|
239 |
## References
|
240 |
* [High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/abs/2112.10752)
|
241 |
* [Source Model Implementation](https://github.com/CompVis/stable-diffusion/tree/main)
|
242 |
|
|
|
|
|
243 |
## Community
|
244 |
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
|
245 |
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
|
|
|
16 |
|
17 |
Generates high resolution images from text prompts using a latent diffusion model. This model uses CLIP ViT-L/14 as text encoder, U-Net based latent denoising, and VAE based decoder to generate the final image.
|
18 |
|
19 |
+
This model is an implementation of Stable-Diffusion-v2.1 found [here]({source_repo}).
|
20 |
This repository provides scripts to run Stable-Diffusion-v2.1 on Qualcomm® devices.
|
21 |
More details on model performance across various devices, can be found
|
22 |
[here](https://aihub.qualcomm.com/models/stable_diffusion_v2_1_quantized).
|
|
|
32 |
- VAE Decoder Number of parameters: 83M
|
33 |
- Model size: 1GB
|
34 |
|
35 |
+
| Model | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
|
36 |
+
|---|---|---|---|---|---|---|---|---|
|
37 |
+
| TextEncoder_Quantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 11.633 ms | 0 - 1 MB | INT8 | NPU | [Stable-Diffusion-v2.1.bin](https://huggingface.co/qualcomm/Stable-Diffusion-v2.1/blob/main/TextEncoder_Quantized.bin) |
|
38 |
+
| TextEncoder_Quantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 7.759 ms | 0 - 8 MB | INT8 | NPU | [Stable-Diffusion-v2.1.bin](https://huggingface.co/qualcomm/Stable-Diffusion-v2.1/blob/main/TextEncoder_Quantized.bin) |
|
39 |
+
| TextEncoder_Quantized | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 11.773 ms | 0 - 0 MB | INT8 | NPU | Use Export Script |
|
40 |
+
| TextEncoder_Quantized | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 10.7 ms | 0 - 1 MB | UINT16 | NPU | Use Export Script |
|
41 |
+
| VAEDecoder_Quantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 217.134 ms | 0 - 2 MB | INT8 | NPU | [Stable-Diffusion-v2.1.bin](https://huggingface.co/qualcomm/Stable-Diffusion-v2.1/blob/main/VAEDecoder_Quantized.bin) |
|
42 |
+
| VAEDecoder_Quantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 161.705 ms | 0 - 8 MB | INT8 | NPU | [Stable-Diffusion-v2.1.bin](https://huggingface.co/qualcomm/Stable-Diffusion-v2.1/blob/main/VAEDecoder_Quantized.bin) |
|
43 |
+
| VAEDecoder_Quantized | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 220.179 ms | 0 - 0 MB | INT8 | NPU | Use Export Script |
|
44 |
+
| VAEDecoder_Quantized | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 225.416 ms | 0 - 2 MB | UINT16 | NPU | Use Export Script |
|
45 |
+
| UNet_Quantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 101.094 ms | 0 - 2 MB | INT8 | NPU | [Stable-Diffusion-v2.1.bin](https://huggingface.co/qualcomm/Stable-Diffusion-v2.1/blob/main/UNet_Quantized.bin) |
|
46 |
+
| UNet_Quantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 72.62 ms | 0 - 8 MB | INT8 | NPU | [Stable-Diffusion-v2.1.bin](https://huggingface.co/qualcomm/Stable-Diffusion-v2.1/blob/main/UNet_Quantized.bin) |
|
47 |
+
| UNet_Quantized | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 102.486 ms | 0 - 0 MB | INT8 | NPU | Use Export Script |
|
48 |
+
| UNet_Quantized | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 96.631 ms | 1 - 2 MB | UINT16 | NPU | Use Export Script |
|
49 |
|
50 |
|
51 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
52 |
|
53 |
## Installation
|
54 |
|
|
|
104 |
```bash
|
105 |
python -m qai_hub_models.models.stable_diffusion_v2_1_quantized.export
|
106 |
```
|
|
|
107 |
```
|
108 |
+
Profiling Results
|
109 |
+
------------------------------------------------------------
|
110 |
+
TextEncoder_Quantized
|
111 |
+
Device : Samsung Galaxy S23 (13)
|
112 |
+
Runtime : QNN
|
113 |
+
Estimated inference time (ms) : 11.6
|
114 |
+
Estimated peak memory usage (MB): [0, 1]
|
115 |
+
Total # Ops : 1040
|
116 |
+
Compute Unit(s) : NPU (1040 ops)
|
117 |
+
|
118 |
+
------------------------------------------------------------
|
119 |
+
VAEDecoder_Quantized
|
120 |
+
Device : Samsung Galaxy S23 (13)
|
121 |
+
Runtime : QNN
|
122 |
+
Estimated inference time (ms) : 217.1
|
123 |
+
Estimated peak memory usage (MB): [0, 2]
|
124 |
+
Total # Ops : 170
|
125 |
+
Compute Unit(s) : NPU (170 ops)
|
126 |
+
|
127 |
+
------------------------------------------------------------
|
128 |
+
UNet_Quantized
|
129 |
+
Device : Samsung Galaxy S23 (13)
|
130 |
+
Runtime : QNN
|
131 |
+
Estimated inference time (ms) : 101.1
|
132 |
+
Estimated peak memory usage (MB): [0, 2]
|
133 |
+
Total # Ops : 6361
|
134 |
+
Compute Unit(s) : NPU (6361 ops)
|
135 |
```
|
136 |
|
137 |
|
|
|
242 |
Get more details on Stable-Diffusion-v2.1's performance across various devices [here](https://aihub.qualcomm.com/models/stable_diffusion_v2_1_quantized).
|
243 |
Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
|
244 |
|
245 |
+
|
246 |
## License
|
247 |
+
* The license for the original implementation of Stable-Diffusion-v2.1 can be found [here](https://github.com/CompVis/stable-diffusion/blob/main/LICENSE).
|
248 |
+
* The license for the compiled assets for on-device deployment can be found [here](https://github.com/CompVis/stable-diffusion/blob/main/LICENSE)
|
249 |
+
|
250 |
+
|
251 |
|
252 |
## References
|
253 |
* [High-Resolution Image Synthesis with Latent Diffusion Models](https://arxiv.org/abs/2112.10752)
|
254 |
* [Source Model Implementation](https://github.com/CompVis/stable-diffusion/tree/main)
|
255 |
|
256 |
+
|
257 |
+
|
258 |
## Community
|
259 |
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
|
260 |
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
|