File size: 660 Bytes
3d594e3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

Qwen2.5-3B-Instruct
Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	755	112.0612			0.97138
IQ1_M	811	42.7456			0.34718
IQ2_XXS	905	25.2117			0.20222
IQ2_XS	984	15.9149			0.11965
IQ2_S	1013	14.5975			0.10820
IQ2_M	1088	12.8779			0.09436
Q2_K_S	1143	13.0878			0.09636
Q2_K	1216	11.8001			0.08674
IQ3_XXS	1224	10.6049			0.07572
IQ3_XS	1328	10.0306			0.06975
Q3_K_S	1387	15.5457			0.11941
IQ3_S	1390	9.9591			0.06984
IQ3_M	1420	9.9957			0.06962
Q3_K_M	1517	14.0989			0.10568
Q3_K_L	1629	13.8579			0.10372
IQ4_XS	1659	9.2935			0.06517
IQ4_NL	1741	9.2824			0.06503
Q4_0	1744	9.4850			0.06626
Q4_K_S	1750	9.2573			0.06485
Q4_K_M	1841	9.2305			0.06475