| license: llama3.1 | |
| language: | |
| - en | |
| inference: false | |
| fine-tuning: false | |
| tags: | |
| - nvidia | |
| - llama3.1 | |
| - exl2 | |
| datasets: | |
| - nvidia/HelpSteer2 | |
| base_model: nvidia/Llama-3.1-Nemotron-70B-Instruct-HF | |
| pipeline_tag: text-generation | |
| library_name: transformers | |
| # Llama-3.1-Nemotron-70B-Instruct - EXL2 4.5bpw | |
| This is a 4.5bpw EXL2 quant of [nvidia/Llama-3.1-Nemotron-70B-Instruct](https://huggingface.co/nvidia/Llama-3.1-Nemotron-70B-Instruct-HF) | |
| Details about the model can be found at the above model page. | |
| ## EXL2 Version | |
| These quants were made with exllamav2 version 0.2.4. Quants made on this version of EXL2 may not work on older versions of the exllamav2 library. | |
| If you have problems loading these models, please update Text Generation WebUI to the latest version. | |