Edit Models filters

Inference Providers

Nebius AI Studio

HF Inference API

Misc

compressed-tensors

Inference Endpoints

text-generation-inference

AutoTrain Compatible

8-bit precision

Mixture of Experts

Misc with no match

4-bit precision

text-embeddings-inference

Carbon Emissions

Models

937

Full-text search

Active filters: compressed-tensors

nm-testing/tinyllama-w8a16-dense-hf-quantizer

Updated Sep 11, 2024 • 57

nm-testing/DeepSeek-Coder-V2-Lite-Instruct-W8A8

Updated Oct 9, 2024 • 50

nm-testing/DeepSeek-V2.5-W4A16

Updated Oct 9, 2024 • 40 • 2

nm-testing/Qwen2-VL-7B-Instruct-FP8-dynamic

Updated Oct 29, 2024 • 115

nm-testing/Phi-3.5-MoE-instruct-FP8

Updated Oct 9, 2024 • 9 • 2

nm-testing/OLMoE-1B-7B-0924-Instruct-FP8

Updated Oct 9, 2024 • 8

nm-testing/Meta-Llama-3.1-8B-Instruct-FP8-hf

Text Generation • Updated Sep 24, 2024 • 31

nm-testing/tinyllama-oneshot-w8a8-channel-dynamic-token-v2-asym

Updated Oct 9, 2024 • 1.11k

nm-testing/Meta-Llama-3-8B-Instruct-W8-Channel-A8-Dynamic-Asym-Per-Token-Test

Updated Oct 9, 2024 • 1.16k • 1

nm-testing/asym-w8w8-int8-static-per-tensor-tiny-llama

Updated Oct 9, 2024 • 1.2k

neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9, 2024 • 27.4k • 2

neuralmagic/Llama-3.2-3B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9, 2024 • 8.86k • 2

neuralmagic/Llama-3.2-1B-Instruct-quantized.w8a8

Text Generation • Updated Oct 16, 2024 • 6.78k • 7

neuralmagic/Llama-3.2-3B-Instruct-quantized.w8a8

Text Generation • Updated Oct 16, 2024 • 5.96k • 1

neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic

Text Generation • Updated Oct 2, 2024 • 43.5k • 8

neuralmagic/Llama-3.2-1B-Instruct-FP8

Text Generation • Updated Oct 16, 2024 • 9.24k • 2

nm-testing/tiny-llama-w4a16-new-key

Updated Sep 26, 2024 • 6

neuralmagic/Llama-3.2-1B-quantized.w8a8

Updated Jan 16 • 2.04k

neuralmagic/Llama-3.2-3B-quantized.w8a8

Updated Oct 9, 2024 • 6

neuralmagic/Llama-3.2-1B-FP8

Updated Oct 9, 2024 • 133

neuralmagic/Phi-3.5-mini-instruct-FP8-KV

Text Generation • Updated Oct 1, 2024 • 216 • 2

horheynm/TinyLlama-1.1B-Chat-v1.0-QuantizedCache

Updated Sep 30, 2024 • 4

nm-testing/NVLM-D-72B-FP8-dynamic

Updated Oct 7, 2024 • 121 • 3

horheynm/Llama-3.2-11B-Vision-Instruct-FP8-Dynamic

Updated Oct 4, 2024 • 16

swapnil7777/llava_epoch4_dermnet-3.5b

Image-Text-to-Text • Updated Oct 7, 2024 • 6

neuralmagic/Qwen2.5-0.5B-quantized.w8a8

Text Generation • Updated Dec 3, 2024 • 35

neuralmagic/Qwen2.5-0.5B-Instruct-quantized.w8a8

Text Generation • Updated Dec 9, 2024 • 120

neuralmagic/Qwen2.5-1.5B-quantized.w8a8

Text Generation • Updated Dec 3, 2024 • 35

neuralmagic/Qwen2.5-7B-quantized.w8a8

Text Generation • Updated Dec 3, 2024 • 90

neuralmagic/Qwen2.5-72B-quantized.w8a8

Text Generation • Updated Dec 3, 2024 • 58