Hugging Face's logo

OPEA
/
DeepSeek-V3-int4-sym-gptq-inc

Safetensors
deepseek_v3
custom_code
4-bit precision
gptq
Model card Files Community
5
cicdatopea commited on
Commit
c495ad5
·
verified ·
1 Parent(s): c004b1b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -27,6 +27,8 @@ model = AutoModelForCausalLM.from_pretrained(
27
  quantized_model_dir,
28
  torch_dtype=torch.float16,
29
  trust_remote_code=True,
 
 
30
  )
31
 
32
  tokenizer = AutoTokenizer.from_pretrained(quantized_model_dir,trust_remote_code=True)
 
27
  quantized_model_dir,
28
  torch_dtype=torch.float16,
29
  trust_remote_code=True,
30
+ device_map="auto"
31
+
32
  )
33
 
34
  tokenizer = AutoTokenizer.from_pretrained(quantized_model_dir,trust_remote_code=True)