Update README.md

README.md

@@ -13,6 +13,38 @@ tags:
 
 ## ***See [our collection](https://huggingface.co/collections/unsloth/deepseek-v3-all-versions-677cf5cfd7df8b7815fc723c) for versions of Deepseek V3 including bf16 and original formats.***
 
+| Quants | Disk Size | Details |
+| ------ | ------- | ------- |
+| [Q2_K_XS](https://huggingface.co/unsloth/DeepSeek-V3-GGUF/tree/main/DeepSeek-V3-Q2_K_XS) | 207GB | Q2 everything, Q4 embed, Q6 lm_head |
+| [Q2_K_L](https://huggingface.co/unsloth/DeepSeek-V3-GGUF/tree/main/DeepSeek-V3-Q2_K_L) | 228GB | Q3 down_proj Q2 rest, Q4 embed, Q6 lm_head |
+| [Q3_K_M](https://huggingface.co/unsloth/DeepSeek-V3-GGUF/tree/main/DeepSeek-V3-Q3_K_M) | 298GB | Standard Q3_K_M |
+| [Q4_K_M](https://huggingface.co/unsloth/DeepSeek-V3-GGUF/tree/main/DeepSeek-V3-Q4_K_M)  | 377GB | Standard Q4_K_M |
+| [Q5_K_M](https://huggingface.co/unsloth/DeepSeek-V3-GGUF/tree/main/DeepSeek-V3-Q5_K_M) | 443GB | Standard Q5_K_M |
+| [Q6_K](https://huggingface.co/unsloth/DeepSeek-V3-GGUF/tree/main/DeepSeek-V3-Q6_K)  | 513GB | Standard Q6_K |
+| [Q8_0](https://huggingface.co/unsloth/DeepSeek-V3-GGUF/tree/main/DeepSeek-V3-Q8_0) | 712GB | Standard Q8_0 |
+
+
+
+1. [Q2_K_XS](https://huggingface.co/unsloth/DeepSeek-V3-GGUF/tree/main/DeepSeek-V3-Q2_K_XS) should run ok in ~40GB of CPU / GPU VRAM with automatic llama.cpp offloading.
+2. Use K quantization (not V quantization)
+3. Do not forget about `<｜User｜>` and `<｜Assistant｜>` tokens! - Or use a chat template formatter
+4. Example with Q5_0 K quantized cache (V quantized cache doesn't work):
+   ```bash
+   ./llama.cpp/llama-cli
+   --model unsloth/DeepSeek-V3-GGUF/DeepSeek-V3-Q2_K_XS/DeepSeek-V3-Q2_K_XS-00001-of-00005.gguf
+   --cache-type-k q5_0
+   --prompt '<｜User｜>What is 1+1?<｜Assistant｜>'
+   ```
+   Example output:
+   ```txt
+   The sum of 1 and 1 is **2**. Here's a simple step-by-step breakdown:
+
+    1. **Start with the number 1.**
+    2. **Add another 1 to it.**
+    3. **The result is 2.**
+
+    So, **1 + 1 = 2**. [end of text]
+    ```
 
 # Finetune Llama 3.3, Gemma 2, Mistral 2-5x faster with 70% less memory via Unsloth!
 We have a free Google Colab Tesla T4 notebook for Llama 3.1 (8B) here: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Llama3.1_(8B)-Alpaca.ipynb