Update README.md
Browse files
README.md
CHANGED
@@ -27,7 +27,7 @@ widget:
|
|
27 |
content: Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.
|
28 |
---
|
29 |
|
30 |
-
This is [Nous Hermes 2 Mistral 7B](https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO), quantized with the help of imatrix so it could
|
31 |
|
32 |
Here's a chart that provides an approximation of the HellaSwag score (out of 1,000 tasks) and the RAM usage (with `--no-mmap`) with llama.cpp.
|
33 |
|Quantization|HellaSwag|256 ctx RAM|512 ctx RAM|1024 ctx RAM|2048 ctx RAM|4096 ctx RAM|8192 ctx RAM
|
|
|
27 |
content: Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.
|
28 |
---
|
29 |
|
30 |
+
This is [Nous Hermes 2 Mistral 7B](https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO), quantized with the help of imatrix so it could offer better performance for being quantized, and have quantization levels available for lower-memory devices to run. [Kalomaze's "groups_merged.txt"](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384) was used for the importance matrix, with context set to 8,192.
|
31 |
|
32 |
Here's a chart that provides an approximation of the HellaSwag score (out of 1,000 tasks) and the RAM usage (with `--no-mmap`) with llama.cpp.
|
33 |
|Quantization|HellaSwag|256 ctx RAM|512 ctx RAM|1024 ctx RAM|2048 ctx RAM|4096 ctx RAM|8192 ctx RAM
|