DevQuasar
/

deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8

Text Generation

text-generation-inference

8-bit precision

Model card Files Files and versions

'Make knowledge free for everyone'

EXPERIMENTAL!

Channel wise INT8 quant. Requires CPU with AMX support (Xeon 5 and above) and 700GB-1TB ram. Not tested due to hardware.

SGlang sould support it. SGlang CPU

Made with https://huggingface.co/meituan/DeepSeek-R1-Channel-INT8/blob/main/inference/bf16_cast_channel_int8.py

Downloads last month: 388

Safetensors

Model size

686B params

Tensor type

F32

·

BF16

·

I8

·

Model tree for DevQuasar/deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8

Base model

deepseek-ai/DeepSeek-V3.2-Exp-Base

Finetuned

deepseek-ai/DeepSeek-V3.2-Speciale

Finetuned

(3)

this model