Hampetiudo
commited on
Commit
•
11a4136
1
Parent(s):
c05770e
Update README.md
Browse files## Llama.cpp imatrix quants of gemma-2-Ifable-9B
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3804">b3804</a> for quantization.
Original model: https://huggingface.co/ifable/gemma-2-Ifable-9B
All quants made using the imatrix option with the dataset from [here](https://gist.github.com/tristandruyen/9e207a95c7d75ddf37525d353e00659c) and with a context size of 8192.
I also uploaded the BF16 GGUF because I'm too lazy to make every single quant and also in case the original model gets taken down.