Intel
/

Qwen2-7B-int4-inc

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

intel/auto-round

Model card Files Files and versions Community

n1ck-guo commited on Sep 29

Commit

327644c

•

1 Parent(s): 8b6797b

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -111,7 +111,7 @@ python3 main.py \
 --iter 1000 \
 --disable_eval \
 --model_dtype "float16" \
---deployment_device 'autoround' \
 --output_dir "./tmp_autoround"
 ```

 --iter 1000 \
 --disable_eval \
 --model_dtype "float16" \
+--deployment_device 'auto_round' \
 --output_dir "./tmp_autoround"
 ```