This is a quantized GGUF of mistralai/Mistral-Nemo-Instruct-2407. Requires llama.cpp newer than commit 50e0535 (7/22/2024) to run inference.

Currently, we just have a Q5_K quantization which comes in at 8.73 GB. If you're interested other quantizations, just ping me @iamlemec on Twitter.

Downloads last month
7
GGUF
Model size
12.2B params
Architecture
llama

5-bit

Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.