SPAHE
/

Meltemi-7B-Instruct-v1-GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

ibalampanis commited on Apr 17

Commit

9605f51

•

1 Parent(s): 523ea36

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -27,6 +27,7 @@ This repository contains GGUF format model files for [ilsp's Meltemi 7B Instruct
 ## Provided files
 | Name                                                                                                                                        | Quantization Method | Precision (Bits) | File Size | Max RAM Required | Use Case                                    |
 | ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | ---------------- | --------- | ---------------- | ------------------------------------------- |
 | [meltemi-7b-instruct-v1_q4_k_m.gguf](https://huggingface.co/SPAHE/Meltemi-7B-Instruct-v1-GGUF/blob/main/meltemi-7b-instruct-v1_q4_k_m.gguf) | Q4_K_M              | 4                | 4.54 GB   | 4.41 GB          | Medium, balanced quality                    |
 | [meltemi-7b-instruct-v1_q6_k.gguf](https://huggingface.co/SPAHE/Meltemi-7B-Instruct-v1-GGUF/blob/main/meltemi-7b-instruct-v1_q6_k.gguf)     | Q6_K                | 6                | 6.14 GB   | 5.92 GB          | Medium, low quality loss                    |
 | [meltemi-7b-instruct-v1_q8_0.gguf](https://huggingface.co/SPAHE/Meltemi-7B-Instruct-v1-GGUF/blob/main/meltemi-7b-instruct-v1_q8_0.gguf)     | Q8_0                | 8                | 7.95 GB   | 7.30 GB          | Large, low quality loss                     |

 ## Provided files
 | Name                                                                                                                                        | Quantization Method | Precision (Bits) | File Size | Max RAM Required | Use Case                                    |
 | ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | ---------------- | --------- | ---------------- | ------------------------------------------- |
+| [meltemi-7b-instruct-v1_q3_k_s.gguf](https://huggingface.co/SPAHE/Meltemi-7B-Instruct-v1-GGUF/blob/main/meltemi-7b-instruct-v1_q3_k_s.gguf) | Q3_K_S              | 3                | 3.32 GB   | 3.69 GB          | Medium, high quality loss                   |
 | [meltemi-7b-instruct-v1_q4_k_m.gguf](https://huggingface.co/SPAHE/Meltemi-7B-Instruct-v1-GGUF/blob/main/meltemi-7b-instruct-v1_q4_k_m.gguf) | Q4_K_M              | 4                | 4.54 GB   | 4.41 GB          | Medium, balanced quality                    |
 | [meltemi-7b-instruct-v1_q6_k.gguf](https://huggingface.co/SPAHE/Meltemi-7B-Instruct-v1-GGUF/blob/main/meltemi-7b-instruct-v1_q6_k.gguf)     | Q6_K                | 6                | 6.14 GB   | 5.92 GB          | Medium, low quality loss                    |
 | [meltemi-7b-instruct-v1_q8_0.gguf](https://huggingface.co/SPAHE/Meltemi-7B-Instruct-v1-GGUF/blob/main/meltemi-7b-instruct-v1_q8_0.gguf)     | Q8_0                | 8                | 7.95 GB   | 7.30 GB          | Large, low quality loss                     |