Hugging Face's logo

Crataco
/
Nous-Hermes-2-Mistral-7B-DPO-imatrix-GGUF

GGUF
English
Mistral
instruct
finetune
chatml
DPO
RLHF
gpt4
synthetic data
distillation
Inference Endpoints
conversational
Model card Files Community
2
Crataco commited on
Commit
92226f2
·
verified ·
1 Parent(s): 380b03e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -4
README.md CHANGED
@@ -32,11 +32,12 @@ This is [Nous Hermes 2 Mistral 7B](https://huggingface.co/NousResearch/Nous-Herm
32
  Here's a chart that provides an approximation of the HellaSwag score (out of 1,000 tasks) and the RAM usage (with `--no-mmap`) with llama.cpp.
33
  |Quantization|HellaSwag|256 ctx RAM|512 ctx RAM|1024 ctx RAM|2048 ctx RAM|4096 ctx RAM|8192 ctx RAM
34
  |--------|--------|--------|--------|--------|--------|--------|--------|
35
- |iq1_S |51.7% |1.6 GiB |1.6 GiB |1.7 GiB |1.8 GiB |2.0 GiB |2.5 GiB |
36
- |iq2_XXS |72.5% |1.9 GiB |1.9 GiB |2.0 GiB |2.1 GiB |2.4 GiB |2.9 GiB |
37
- |iq2_XS |74.2% |2.1 GiB |2.1 GiB |2.2 GiB |2.3 GiB |2.6 GiB |3.1 GiB |
38
- |iq2_S |76.8% |2.2 GiB |2.2 GiB |2.3 GiB |2.4 GiB |2.7 GiB |3.2 GiB |
39
  |q2_K |77.4% |2.6 GiB |2.6 GiB |2.7 GiB |2.8 GiB |3.1 GiB |3.6 GiB |
 
40
  |q3_K_M |80.0% |3.3 GiB |3.4 GiB |3.4 GiB |3.6 GiB |3.8 GiB |4.3 GiB |
41
  |q4_K_M |81.8% |4.1 GiB |4.2 GiB |4.2 GiB |4.3 GiB |4.6 GiB |5.1 GiB |
42
  |q5_K_M |82.1% |4.8 GiB |4.9 GiB |4.9 GiB |5.1 GiB |5.3 GiB |5.8 GiB |
 
32
  Here's a chart that provides an approximation of the HellaSwag score (out of 1,000 tasks) and the RAM usage (with `--no-mmap`) with llama.cpp.
33
  |Quantization|HellaSwag|256 ctx RAM|512 ctx RAM|1024 ctx RAM|2048 ctx RAM|4096 ctx RAM|8192 ctx RAM
34
  |--------|--------|--------|--------|--------|--------|--------|--------|
35
+ |iq1_S (imatrix)|51.7% |1.6 GiB |1.6 GiB |1.7 GiB |1.8 GiB |2.0 GiB |2.5 GiB |
36
+ |iq2_XXS (imatrix)|72.5% |1.9 GiB |1.9 GiB |2.0 GiB |2.1 GiB |2.4 GiB |2.9 GiB |
37
+ |iq2_XS (imatrix)|74.2% |2.1 GiB |2.1 GiB |2.2 GiB |2.3 GiB |2.6 GiB |3.1 GiB |
38
+ |iq2_S (imatrix)|76.8% |2.2 GiB |2.2 GiB |2.3 GiB |2.4 GiB |2.7 GiB |3.2 GiB |
39
  |q2_K |77.4% |2.6 GiB |2.6 GiB |2.7 GiB |2.8 GiB |3.1 GiB |3.6 GiB |
40
+ |q2_K (imatrix)|78.7%
41
  |q3_K_M |80.0% |3.3 GiB |3.4 GiB |3.4 GiB |3.6 GiB |3.8 GiB |4.3 GiB |
42
  |q4_K_M |81.8% |4.1 GiB |4.2 GiB |4.2 GiB |4.3 GiB |4.6 GiB |5.1 GiB |
43
  |q5_K_M |82.1% |4.8 GiB |4.9 GiB |4.9 GiB |5.1 GiB |5.3 GiB |5.8 GiB |