Hugging Face's logo

Nexesenex
/
Meta_Llama-3.1-8b-it_iMat_Custom_Quant_Stategies-GGUF

GGUF
Inference Endpoints
imatrix
conversational
Model card Files Community
1
Nexesenex commited on
Commit
0568bae
•
1 Parent(s): 6c2b64e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +33 -21
README.md CHANGED
@@ -18,84 +18,96 @@ ARC and PPL-512 DATA (Get the last data on the main post of the PR thread) :
18
  ```
19
  IQ1_XS
20
  PR
 
 
21
 
22
-
23
 
24
 
25
  IQ1_S
26
  Master
27
- 1.87 GiB (2.00 BPW)
28
  2.01 GB (2.00 BPW)
 
29
  PPL over 564 chunks for n_ctx=512 = 61.2817 +/- 0.41707
30
 
31
  PR
32
-
33
-
 
34
 
35
 
36
  IQ1_M
37
  Master
38
- 2.01 GiB (2.15 BPW)
39
  2.15 GB (2.15 BPW)
 
40
  PPL over 564 chunks for n_ctx=512 = 26.3761 +/- 0.18200
41
 
42
  PR
43
-
44
-
 
45
 
46
 
47
  IQ1_XL
48
  PR
49
-
50
-
51
-
52
-
53
 
54
 
55
  IQ2_XXS
56
  Master
57
- 2.23 GiB (2.38 BPW)
58
  2.39 GB (2.38 BPW)
 
59
  PPL over 564 chunks for n_ctx=512 = 15.2572 +/- 0.10267
60
 
61
  PR
 
 
 
62
 
63
 
64
  IQ2_XS
65
  Master
66
- 2.42 GiB (2.59 BPW)
67
  2.60 GB (2.59 BPW)
 
68
  PPL over 564 chunks for n_ctx=512 = 11.7483 +/- 0.07776
69
 
70
  PR
 
 
 
71
 
72
 
73
  IQ2_S
74
  Master
75
- 2.56 GiB (2.74 BPW)
76
  2.75 GB (2.74 BPW)
 
77
  PPL over 564 chunks for n_ctx=512 = 10.5180 +/- 0.06976
78
 
79
  PR
 
 
 
80
 
81
 
82
  IQ2_M
83
  Master
84
- 2.74 GiB (2.93 BPW)
85
  2.94 GB (2.93 BPW)
 
86
  PPL over 564 chunks for n_ctx=512 = 9.5935 +/- 0.06228
87
 
88
  PR
89
-
90
-
91
-
92
 
93
 
94
  IQ2_XL
95
  PR
96
-
97
-
98
-
99
 
100
 
101
  IQ3_XXS
 
18
  ```
19
  IQ1_XS
20
  PR
21
+ 1.94 GB (1.93 BPW)
22
+ 1.81 GiB (1.93 BPW)
23
 
24
+ PPL over 564 chunks for n_ctx=512 = 40.0024 +/- 0.27710
25
 
26
 
27
  IQ1_S
28
  Master
 
29
  2.01 GB (2.00 BPW)
30
+ 1.87 GiB (2.00 BPW)
31
  PPL over 564 chunks for n_ctx=512 = 61.2817 +/- 0.41707
32
 
33
  PR
34
+ 2.05 GB (2.04 BPW)
35
+ 1.91 GiB (2.04 BPW)
36
+ PPL over 564 chunks for n_ctx=512 = 25.2524 +/- 0.17651
37
 
38
 
39
  IQ1_M
40
  Master
 
41
  2.15 GB (2.15 BPW)
42
+ 2.01 GiB (2.15 BPW)
43
  PPL over 564 chunks for n_ctx=512 = 26.3761 +/- 0.18200
44
 
45
  PR
46
+ 2.14 GB (2.13 BPW)
47
+ 1.99 GiB (2.13 BPW)
48
+ PPL over 564 chunks for n_ctx=512 = 20.0588 +/- 0.14001
49
 
50
 
51
  IQ1_XL
52
  PR
53
+ 2.21 GB (2.21 BPW)
54
+ 2.06 GiB (2.21 BPW)
55
+ PPL over 564 chunks for n_ctx=512 = 18.5500 +/- 0.12753
 
56
 
57
 
58
  IQ2_XXS
59
  Master
 
60
  2.39 GB (2.38 BPW)
61
+ 2.23 GiB (2.38 BPW)
62
  PPL over 564 chunks for n_ctx=512 = 15.2572 +/- 0.10267
63
 
64
  PR
65
+ 2.38 GB (2.37 BPW)
66
+ 2.22 GiB (2.37 BPW)
67
+ PPL over 564 chunks for n_ctx=512 = 13.8073 +/- 0.09290
68
 
69
 
70
  IQ2_XS
71
  Master
 
72
  2.60 GB (2.59 BPW)
73
+ 2.42 GiB (2.59 BPW)
74
  PPL over 564 chunks for n_ctx=512 = 11.7483 +/- 0.07776
75
 
76
  PR
77
+ 2.52 GB (2.51 BPW)
78
+ 2.35 GiB (2.51 BPW)
79
+ PPL over 564 chunks for n_ctx=512 = 11.6639 +/- 0.07805
80
 
81
 
82
  IQ2_S
83
  Master
 
84
  2.75 GB (2.74 BPW)
85
+ 2.56 GiB (2.74 BPW)
86
  PPL over 564 chunks for n_ctx=512 = 10.5180 +/- 0.06976
87
 
88
  PR
89
+ 2.71 GB (2.70 BPW)
90
+ 2.52 GiB (2.70 BPW)
91
+ PPL over 564 chunks for n_ctx=512 = 10.7010 +/- 0.07027
92
 
93
 
94
  IQ2_M
95
  Master
 
96
  2.94 GB (2.93 BPW)
97
+ 2.74 GiB (2.93 BPW)
98
  PPL over 564 chunks for n_ctx=512 = 9.5935 +/- 0.06228
99
 
100
  PR
101
+ 2.93 GB (2.92 BPW)
102
+ 2.73 GiB (2.92 BPW)
103
+ PPL over 564 chunks for n_ctx=512 = 9.4125 +/- 0.06039
104
 
105
 
106
  IQ2_XL
107
  PR
108
+ 2.99 GB (2.98 BPW)
109
+ 2.78 GiB (2.98 BPW)
110
+ PPL over 564 chunks for n_ctx=512 = 9.3122 +/- 0.05973
111
 
112
 
113
  IQ3_XXS