Edit Models filters

Inference status

Misc

compressed-tensors

AutoTrain Compatible

Inference Endpoints

text-generation-inference

8-bit precision

Misc with no match

4-bit precision

text-embeddings-inference

Carbon Emissions

Mixture of Experts

Models

503

Full-text search

Active filters: compressed-tensors

neuralmagic/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-dynamic

Text Generation • Updated Oct 17 • 15.6k • 14

thwin27/Aria-sequential_mlp-FP8-dynamic

Image-Text-to-Text • Updated Oct 23 • 82 • 5

nm-testing/Qwen2.5-Coder-32B-Instruct-FP8-dynamic

Updated Nov 15 • 520 • 1

nm-testing/Qwen2.5-32B-Instruct-FP8-dynamic

Updated Nov 15 • 153 • 1

nm-testing/pixtral-12b-W8A8

Updated 28 days ago • 177 • 2

neuralmagic/Sparse-Llama-3.1-8B-ultrachat_200k-2of4-quantized.w4a16

Text Generation • Updated 26 days ago • 2.01k • 2

qeternity/AIDC-AI_Marco-o1-GPTQ-INT4

Updated 21 days ago • 36 • 1

reinforce20001/SakuraLLM.Sakura-14B-Qwen2.5-v1.0-W8A8-Int8

Updated 20 days ago • 75 • 1

pL-Community/SauerkrautLM-v2-14b-DPO-FP8

Text Generation • Updated 14 days ago • 94 • 1

cortecs/Llama-3.3-70B-Instruct-FP8-Dynamic

Updated 5 days ago • 476 • 3

cortecs/EuroLLM-9B-Instruct-FP8-Dynamic

Updated 5 days ago • 90 • 1

CalamitousFelicitousness/Evathene-v1.0-FP8-Dynamic

Text Generation • Updated 28 days ago • 27 • 2

nm-testing/tinyllama-one-shot-static-quant-test-compressed

Text Generation • Updated Oct 9 • 9

nm-testing/tinyllama-one-shot-dynamic-test

Text Generation • Updated Oct 9 • 10

nm-testing/tinyllama-one-shot-w4a16-group-packed

Text Generation • Updated Oct 10 • 23

nm-testing/tinyllama-one-shot-w4a16-channel-compressed

Text Generation • Updated Oct 9 • 13

nm-testing/tinyllama-one-shot-w4a16-channel-packed

Text Generation • Updated Oct 9 • 12

nm-testing/llama7b-one-shot-2_4-w4a16-packed

Text Generation • Updated Oct 9 • 12

nm-testing/tinyllama-one-shot-w4a16-group128-packed

Text Generation • Updated Oct 9 • 13

nm-testing/llama3-8b-w8_channel-a8_tensor-compressed

Text Generation • Updated Oct 9 • 11

nm-testing/llama7b-one-shot-2_4-w4a16-marlin24

Text Generation • Updated Jun 4 • 12

nm-testing/llama7b-one-shot-2_4-w4a16-group128-packed

Text Generation • Updated Jun 4 • 13

nm-testing/llama1.1b_0.5_sparse_bitmask

Text Generation • Updated Oct 9 • 9

nm-testing/llama7b-one-shot-2_4-w4a16-marlin24-t

Text Generation • Updated Oct 9 • 5.43k • 1

nm-testing/tinyllama-one-shot-w8a8-dynamic-channel

Text Generation • Updated Oct 9 • 9

nm-testing/llama7b-one-shot-2_4-w4a16-marlin24-t-alt

Text Generation • Updated Oct 9 • 13

nm-testing/tinyllama-marlin24-w4a16-group128

Text Generation • Updated Oct 9 • 11

nm-testing/tinyllama-oneshot-w8a8-static-v2

Text Generation • Updated Oct 9 • 9

nm-testing/tinyllama-oneshot-w8a8-dynamic-token-v2

Text Generation • Updated Oct 9 • 6.84k

nm-testing/tinyllama-oneshot-w8a8-static-v3

Text Generation • Updated Jun 17 • 403