metadata
license: apache-2.0
This is a quantized GGUF of mistralai/Mistral-Nemo-Instruct-2407.
Right now, to run it in llama.cpp you'll need to use PR #8577 or equivalently the fork iamlemec/llama.cpp.
Currently, we just have a Q5_K quantization which comes in at 8.73 GB. If you're interested other quantizations, just ping me @iamlemec on Twitter.