This model is only for testing. It's a reupload of nm-testing/Llama-3_1-8B_2of4_w4a16_gsm8k_256_8196_damp0_1_mse_llm_compressor, renamed to fit into Linux' path length for Unix domain sockets.

Safetensors

Model size

1.98B params

Tensor type

I32

BF16

FP16

I16

Inference Providers NEW

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The model has no library tag.

Model tree for danieldk/Llama-3.1-8B-w4a16-int-24

Base model

Quantized

(186)

this model