Edit Models filters

Inference status

Misc

compressed-tensors

AutoTrain Compatible

Inference Endpoints

text-generation-inference

8-bit precision

Misc with no match

4-bit precision

text-embeddings-inference

Carbon Emissions

Mixture of Experts

Models

503

Full-text search

Active filters: compressed-tensors

neuralmagic/gemma-2-2b-it-FP8

Updated Oct 9 • 2.65k • 1

nm-testing/Mixtral-8x7B-Instruct-v0.1-W8A8-quantized

Updated Oct 9 • 3

nm-testing/Mixtral-8x7B-Instruct-v0.1-W4A16-quantized

Updated Oct 9 • 128

nm-testing/Mixtral-8x7B-Instruct-v0.1-W8A16-quantized

Updated Oct 9 • 159

nm-testing/Phi-3-mini-128k-instruct-FP8

Updated Oct 9 • 7.02k

nm-testing/Meta-Llama-3-8B-Instruct-fp8-compressed

Updated Oct 9 • 3

neuralmagic/gemma-2-9b-it-quantized.w8a16

Text Generation • Updated Oct 9 • 91 • 1

neuralmagic/gemma-2-2b-it-quantized.w8a16

Text Generation • Updated Oct 9 • 47 • 1

neuralmagic/gemma-2-2b-quantized.w8a16

Text Generation • Updated Oct 9 • 37

nm-testing/Mixtral-8x7B-Instruct-v0.1-W4A16-channel-quantized

Updated Oct 9 • 129

neuralmagic/SmolLM-1.7B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9 • 21

neuralmagic/gemma-2-2b-it-quantized.w8a8

Text Generation • Updated Oct 9 • 2.13k

neuralmagic/gemma-2-9b-it-quantized.w8a8

Text Generation • Updated Oct 9 • 559 • 2

neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a8

Text Generation • Updated 15 days ago • 369 • 2

neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9 • 362 • 2

nm-testing/Meta-Llama-3-8B-Instruct-fp8-hf_compat

Updated Aug 30 • 37

neuralmagic/SmolLM-360M-Instruct-quantized.w8a8

Text Generation • Updated Oct 9 • 42

neuralmagic/SmolLM-135M-Instruct-quantized.w8a8

Text Generation • Updated Oct 9 • 342

nm-testing/SmolLM-135M-Instruct-quantized.w4a16

Text Generation • Updated Oct 9 • 8

nm-testing/SmolLM-360M-Instruct-quantized.w4a16

Updated Oct 9 • 3

nm-testing/SmolLM-1.7B-Instruct-quantized.w4a16

Text Generation • Updated Oct 9 • 6

nm-testing/TinyLlama-1.1B-Chat-v1.0-W4A16_channel-e2e

Text Generation • Updated about 5 hours ago • 448

nm-testing/TinyLlama-1.1B-Chat-v1.0-W4A16-e2e

Text Generation • Updated about 4 hours ago • 403

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A16_channel-e2e

Text Generation • Updated about 4 hours ago • 458

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A16-e2e

Text Generation • Updated about 4 hours ago • 421

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8A16_channel-e2e

Text Generation • Updated about 5 hours ago • 31

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8A16_tensor-e2e

Text Generation • Updated about 5 hours ago • 27

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8-e2e

Text Generation • Updated about 4 hours ago • 516

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8_DYNAMIC-e2e

Text Generation • Updated about 5 hours ago • 53

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A8-e2e

Text Generation • Updated about 5 hours ago • 34