---
license: llama3.1
tags:
- llama
- conversational
- text-generation-inference
- facebook
- meta
- llama-3
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
library_name: transformers
---

> [!IMPORTANT]
> **NOTICE:**<br>
> Llama-3.1 is licensed under [Llama 3.1 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)<br>
> A copy of this license is available at this repo, [here](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/LICENSE)

**Original Model:** [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct)

**How to Use:** [llama.cpp](https://github.com/ggerganov/llama.cpp)

**Original Model License:** Llama 3.1 Community License

**Release Used:** [b3441](https://github.com/ggerganov/llama.cpp/releases/tag/b3441)

# Quants
| Name | Quant Type | Size |
| ---- | ---- | ---- |
| [Meta-Llama-3.1-8B-Instruct-Q2_K.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q2_K.gguf) | Q2_K | 3.18 GB |
| [Meta-Llama-3.1-8B-Instruct-Q3_K_S.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q3_K_S.gguf) | Q3_K_S | 3.66 GB |
| [Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf) | Q3_K_M | 4.02 GB |
| [Meta-Llama-3.1-8B-Instruct-Q3_K_L.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q3_K_L.gguf) | Q3_K_L | 4.32 GB |
| [Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf) | Q4_K_S | 4.69 GB |
| [Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf) | Q4_K_M | 4.92 GB |
| [Meta-Llama-3.1-8B-Instruct-Q5_K_S.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q5_K_S.gguf) | Q5_K_S | 5.60 GB |
| [Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf) | Q5_K_M | 5.73 GB |
| [Meta-Llama-3.1-8B-Instruct-Q6_K.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q6_K.gguf) | Q6_K | 6.60 GB |
| [Meta-Llama-3.1-8B-Instruct-Q8_0.gguf](https://huggingface.co/starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf) | Q8_0 | 8.54 GB |