Edit Models filters

Inference status

Misc

4-bit precision

AutoTrain Compatible

text-generation-inference

Inference Endpoints

8-bit precision

Mixture of Experts

text-embeddings-inference

Carbon Emissions

Models

16,830

Full-text search

Active filters: 4-bit

unsloth/gemma-2-9b-it-bnb-4bit

Text Generation • Updated Sep 3 • 25.8k • 19

unsloth/gemma-2-27b-it-bnb-4bit

Text Generation • Updated Sep 3 • 5.04k • 7

marcsun13/gemma-2-9b-it-GPTQ

Text Generation • Updated Jul 3 • 3.13k • 3

TechxGenus/DeepSeek-V2-Lite-Chat-AWQ

Text Generation • Updated Jul 4 • 202 • 1

Shaleen123/gemma2-9b-medical

Text Generation • Updated Jul 4 • 75 • 3

model-scope/glm-4-9b-chat-GPTQ-Int4

Text Generation • Updated Jul 17 • 51 • 6

MaziyarPanahi/Llama-3-Groq-8B-Tool-Use-GGUF

Text Generation • Updated Jul 17 • 953k • 5

hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4

Text Generation • Updated Sep 13 • 3.98k • 37

MaziyarPanahi/Mistral-Nemo-Instruct-2407-GGUF

Text Generation • Updated Jul 22 • 963k • 30

unsloth/Mistral-Nemo-Base-2407-bnb-4bit

Text Generation • Updated Sep 11 • 8.3k • 10

Ichigo2899/mistralai-Mistral-Nemo-Instruct-2407-AWQ

Text Generation • Updated Jul 18 • 103 • 1

hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 393k • 60

rinna/llama-3-youko-8b-instruct-gptq

Text Generation • Updated Aug 31 • 104 • 1

shuyuej/gemma-2-27b-it-GPTQ

Text Generation • Updated Sep 28 • 322 • 1

casperhansen/mistral-nemo-instruct-2407-awq

Text Generation • Updated Sep 27 • 226k • 7

shuyuej/Meta-Llama-3.1-70B-Instruct-GPTQ

Text Generation • Updated Sep 28 • 284 • 3

unsloth/Meta-Llama-3.1-8B-bnb-4bit

Text Generation • Updated Sep 3 • 257k • 79

unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit

Text Generation • Updated Sep 11 • 462k • 37

MaziyarPanahi/Meta-Llama-3.1-70B-Instruct-GGUF

Text Generation • Updated Jul 29 • 963k • 35

shuyuej/Mistral-Nemo-Instruct-2407-GPTQ

Text Generation • Updated Jul 25 • 130k • 4

hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 48.5k • 17

hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 52.9k • 19

tiiuae/falcon-mamba-7b-4bit

Text Generation • Updated about 1 month ago • 163 • 11

unsloth/Mistral-Large-Instruct-2407-bnb-4bit

Text Generation • Updated Sep 11 • 327 • 9

shuyuej/Mistral-Large-Instruct-2407-GPTQ

Text Generation • Updated Jul 25 • 314 • 3

TechxGenus/Mistral-Large-Instruct-2407-AWQ

Text Generation • Updated about 1 month ago • 6.97k • 9

casperhansen/mistral-large-instruct-2407-awq

Text Generation • Updated Jul 25 • 620 • 4

dahara1/llama3.1-8b-Instruct-awq

Text Generation • Updated Jul 29 • 205 • 3

neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16

Text Generation • Updated 17 days ago • 14.3k • 21

lucyknada/NeverSleep_Lumimaid-v0.2-12B-exl2-4.0bpw

Text Generation • Updated Jul 26 • 19 • 1