|
--- |
|
base_model: black-forest-labs/FLUX.1-schnell |
|
license: apache-2.0 |
|
language: |
|
- en |
|
pipeline_tag: text-to-image |
|
tags: |
|
- text-to-image |
|
- image-generation |
|
- flux |
|
--- |
|
|
|
Quantized versions of https://huggingface.co/black-forest-labs/FLUX.1-schnell |
|
|
|
Tools used for quantization: modded [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp), [LlamaQuantizer](https://github.com/aifoundry-org/LlamaQuantizer) |
|
|
|
**Work in progress, use at your own risk** |
|
|
|
## How to: |
|
[WIP] |
|
|
|
1. Dowload and build [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp), `flux` branch |
|
2. Download one of the models from this repo and |
|
* Autoencoder https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/ae.safetensors |
|
* CLIP_L https://huggingface.co/comfyanonymous/flux_text_encoders/blob/main/clip_l.safetensors |
|
* T5XXL https://huggingface.co/comfyanonymous/flux_text_encoders/blob/main/t5xxl_fp16.safetensors |
|
3. Enter your `stable-diffusion.cpp` dir |
|
4. Run the following command: |
|
``` |
|
./build/bin/sd --diffusion-model [path to gguf] --vae [path to ae.safetensors] --clip_l [path to clip_l.safetensors] --t5xxl [path to t5xxl_fp16.safetensors] -p "a frog holding a sign saying 'hi' " -o ../frog.png -v --cfg-scale 1.0 --sampling-method euler -v --seed 42 --steps 4 |
|
``` |
|
|
|
|
|
## Results: |
|
|
|
<table style="border-collapse: collapse; width: 100%;"> |
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"><strong>Quant type</strong></td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"><strong>Size</strong></td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"><strong>Result (x0.5)</strong></td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"><strong>Download link</strong></td> |
|
</tr> |
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>default</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>23.8 GB</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/flux_frog_default.png"> |
|
<img src="./examples/flux_frog_default.png" alt="flux_frog_default.png" style="display: block; margin: 0 auto; width: 256px; height: 256px;"> |
|
</a> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/flux1-schnell.safetensors">flux1-schnell.safetensors.gguf</a> |
|
</td> |
|
</tr> |
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>FP16</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong> 23.8 GB</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/flux_frog_F16.png"> |
|
<img src="./examples/flux_frog_F16.png" alt="flux_frog_F16.png" style="display: block; margin: 0 auto; width: 256px; height: 256px;"> |
|
</a> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-F16.gguf">flux1-schnell-F16.gguf</a> |
|
</td> |
|
</tr> |
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>Q8_0</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong> 12.6 GB</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/flux_frog_Q8_0.png"> |
|
<img src="./examples/flux_frog_Q8_0.png" alt="flux_frog_Q8_0.png" style="display: block; margin: 0 auto; width: 256px; height: 256px;"> |
|
</a> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q8_0.gguf">flux1-schnell-Q8_0.gguf</a> |
|
</td> |
|
|
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>Q5_0</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong> 8.18 GB</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/flux_frog_Q5_0.png"> |
|
<img src="./examples/flux_frog_Q5_0.png" alt="flux_frog_Q5_0.png" style="display: block; margin: 0 auto; width: 256px; height: 256px;"> |
|
</a> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q5_0.gguf">flux1-schnell-Q5_0.gguf</a> |
|
</td> |
|
</tr> |
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>Q5_1</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong> 8.92 GB</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/flux_frog_Q5_1.png"> |
|
<img src="./examples/flux_frog_Q5_1.png" alt="flux_frog_Q5_1.png" style="display: block; margin: 0 auto; width: 256px; height: 256px;"> |
|
</a> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q5_1.gguf">flux1-schnell-Q5_1.gguf</a> |
|
</td> |
|
</tr> |
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>Q4_0</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong> 6.69 GB</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/flux_frog_Q4_0.png"> |
|
<img src="./examples/flux_frog_Q4_0.png" alt="flux_frog_Q4_0.png" style="display: block; margin: 0 auto; width: 256px; height: 256px;"> |
|
</a> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q4_0.gguf">flux1-schnell-Q4_0.gguf</a> |
|
</td> |
|
</tr> |
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>Q4_1</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong> 7.43 GB</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/flux_frog_Q4_1.png"> |
|
<img src="./examples/flux_frog_Q4_1.png" alt="flux_frog_Q4_1.png" style="display: block; margin: 0 auto; width: 256px; height: 256px;"> |
|
</a> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q4_1.gguf">flux1-schnell-Q4_1.gguf</a> |
|
</td> |
|
</tr> |
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>Q4_K</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong> 6.69 GB</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/flux_frog_Q4_K.png"> |
|
<img src="./examples/flux_frog_Q4_K.png" alt="flux_frog_Q4_K.png" style="display: block; margin: 0 auto; width: 256px; height: 256px;"> |
|
</a> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q4_K.gguf">flux1-schnell-Q4_K.gguf</a> |
|
</td> |
|
</tr> |
|
<tr> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong>Q2_K</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<strong> 3.9 GB</strong> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/blob/main/examples/flux_frog_Q2_K.png"> |
|
<img src="./examples/flux_frog_Q2_K.png" alt="flux_frog_Q2_K.png" style="display: block; margin: 0 auto; width: 256px; height: 256px;"> |
|
</a> |
|
</td> |
|
<td style="border: none; padding: 10px; text-align: center; vertical-align: middle; font-size: 1.5em;"> |
|
<a href="https://huggingface.co/aifoundry-org/FLUX.1-schnell-Quantized/resolve/main/flux1-schnell-Q2_K.gguf">flux1-schnell-Q2_K.gguf</a> |
|
</td> |
|
</tr> |
|
|
|
</table> |
|
|
|
|
