Hugging Face's logo

csdc-atl
/
Baichuan2-13B-Chat-GPTQ-Int4

Text Generation
Transformers
Safetensors
English
Chinese
baichuan
custom_code
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community
xiaotinghe commited on
Commit
2cc3c7f
•
1 Parent(s): d626856

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -32,7 +32,7 @@ tasks:
32
  | 模型版本 | agieval | ceval | cmmlu | size | 推理速度(A100-40G) |
33
  |---|---|---|---|---|---|
34
  | [Baichuan2-13B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat) | 40.25 | 56.33 | 58.44 | 27.79g | 31.55 tokens/s |
35
- | [Baichuan2-13B-Chat-4bits](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat-4bits) | ~ | ~ | ~ | 9.08g | 18.45 tokens/s |
36
  | [GPTQ-4bit-32g](https://huggingface.co/csdc-atl/Baichuan2-13B-Chat-GPTQ-Int4/tree/4bit-32g) | 38.64 | 57.18 | 57.47 | 9.87g | 27.35(hf) \ 38.28(autogptq) tokens/s |
37
  | [GPTQ-4bit-128g](https://huggingface.co/csdc-atl/Baichuan2-13B-Chat-GPTQ-Int4/tree/main) | 38.78 | 56.42 | 57.78 | 9.14g | 28.74(hf) \ 39.24(autogptq) tokens/s |
38
 
 
32
  | 模型版本 | agieval | ceval | cmmlu | size | 推理速度(A100-40G) |
33
  |---|---|---|---|---|---|
34
  | [Baichuan2-13B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat) | 40.25 | 56.33 | 58.44 | 27.79g | 31.55 tokens/s |
35
+ | [Baichuan2-13B-Chat-4bits](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat-4bits) | 39.01 | 56.63 | 57.81 | 9.08g | 18.45 tokens/s |
36
  | [GPTQ-4bit-32g](https://huggingface.co/csdc-atl/Baichuan2-13B-Chat-GPTQ-Int4/tree/4bit-32g) | 38.64 | 57.18 | 57.47 | 9.87g | 27.35(hf) \ 38.28(autogptq) tokens/s |
37
  | [GPTQ-4bit-128g](https://huggingface.co/csdc-atl/Baichuan2-13B-Chat-GPTQ-Int4/tree/main) | 38.78 | 56.42 | 57.78 | 9.14g | 28.74(hf) \ 39.24(autogptq) tokens/s |
38