Crataco commited on
Commit
31d6f59
1 Parent(s): 2be1252

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -27,7 +27,7 @@ widget:
27
  content: Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.
28
  ---
29
 
30
- This is [Nous Hermes 2 Mistral 7B](https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO), quantized with the help of imatrix so it could run on lower-memory devices. [Kalomaze's "groups_merged.txt"](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384) was used for the importance matrix, with context set to 8,192.
31
 
32
  Here's a chart that provides an approximation of the HellaSwag score (out of 1,000 tasks) and the RAM usage (with `--no-mmap`) with llama.cpp.
33
  |Quantization|HellaSwag|256 ctx RAM|512 ctx RAM|1024 ctx RAM|2048 ctx RAM|4096 ctx RAM|8192 ctx RAM
 
27
  content: Write a short story about Goku discovering kirby has teamed up with Majin Buu to destroy the world.
28
  ---
29
 
30
+ This is [Nous Hermes 2 Mistral 7B](https://huggingface.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO), quantized with the help of imatrix so it could offer better performance for being quantized, and have quantization levels available for lower-memory devices to run. [Kalomaze's "groups_merged.txt"](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384) was used for the importance matrix, with context set to 8,192.
31
 
32
  Here's a chart that provides an approximation of the HellaSwag score (out of 1,000 tasks) and the RAM usage (with `--no-mmap`) with llama.cpp.
33
  |Quantization|HellaSwag|256 ctx RAM|512 ctx RAM|1024 ctx RAM|2048 ctx RAM|4096 ctx RAM|8192 ctx RAM