Crataco
/

Nous-Hermes-2-Mistral-7B-DPO-imatrix-GGUF

Inference Endpoints

Model card Files Files and versions Community

Crataco commited on Apr 10

Commit

9d48d41

•

1 Parent(s): 6e868d9

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -33,6 +33,7 @@ Here's a chart that provides an approximation of the HellaSwag score (out of 1,0
 |Quantization|HellaSwag|256 ctx RAM|512 ctx|1024 ctx|2048 ctx|4096 ctx|8192 ctx
 |--------|--------|--------|--------|--------|--------|--------|--------|
 |IQ1_S   |51.7%   |1.6 GiB |1.6 GiB |1.7 GiB |1.8 GiB |2.0 GiB |2.5 GiB |
 |IQ2_XXS |72.5%   |1.9 GiB |1.9 GiB |2.0 GiB |2.1 GiB |2.4 GiB |2.9 GiB |
 |IQ2_XS  |74.2%   |2.1 GiB |2.1 GiB |2.2 GiB |2.3 GiB |2.6 GiB |3.1 GiB |
 |IQ2_S   |76.8%   |2.2 GiB |2.2 GiB |2.3 GiB |2.4 GiB |2.7 GiB |3.2 GiB |

 |Quantization|HellaSwag|256 ctx RAM|512 ctx|1024 ctx|2048 ctx|4096 ctx|8192 ctx
 |--------|--------|--------|--------|--------|--------|--------|--------|
 |IQ1_S   |51.7%   |1.6 GiB |1.6 GiB |1.7 GiB |1.8 GiB |2.0 GiB |2.5 GiB |
+|IQ1_M   |60.5%   |
 |IQ2_XXS |72.5%   |1.9 GiB |1.9 GiB |2.0 GiB |2.1 GiB |2.4 GiB |2.9 GiB |
 |IQ2_XS  |74.2%   |2.1 GiB |2.1 GiB |2.2 GiB |2.3 GiB |2.6 GiB |3.1 GiB |
 |IQ2_S   |76.8%   |2.2 GiB |2.2 GiB |2.3 GiB |2.4 GiB |2.7 GiB |3.2 GiB |