Hugging Face's logo

Monero
/
Pygmalion-Metharme-7b-4bit-TopScore

Text Generation
Transformers
English
llama
text generation
instruct
Model card Files Community
4
Monero commited on
Commit
7eb77ab
•
1 Parent(s): c8b76cd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1677 -93
README.md CHANGED
@@ -17,7 +17,7 @@ Converted from the XORs weights from PygmalionAI's release https://huggingface.c
17
 
18
  Quantized for KoboldAI use using https://github.com/0cc4m/GPTQ-for-LLaMa
19
 
20
- I created several quantized variations of this model and believe this variation to be "best." <br>
21
  <html>
22
  <head>
23
  <style>
@@ -46,7 +46,7 @@ I created several quantized variations of this model and believe this variation
46
  <thead>
47
  <tr>
48
  <th>Model:</th>
49
- <th>Wikitext 2</th>
50
  <th>Ptb-New</th>
51
  <th>C4-New</th>
52
  </tr>
@@ -58,12 +58,12 @@ I created several quantized variations of this model and believe this variation
58
  <td>41.61103439331055</td>
59
  <td>7.512405872344971</td>
60
  </tr>
61
- <tr>
62
- <td>Metharme 7b - 4bit --act-order</td>
63
- <td><span style="font-family: &quot;Source Sans Pro&quot;, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, &quot;Helvetica Neue&quot;, Arial, &quot;Noto Sans&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;, &quot;Segoe UI Symbol&quot;, &quot;Noto Color Emoji&quot;; font-size: 14.7px; font-style: normal; font-weight: 400;">6.281311511993408</span></td>
64
- <td>46.79158401489258</td>
65
- <td><span style="font-family: &quot;Source Sans Pro&quot;, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, &quot;Helvetica Neue&quot;, Arial, &quot;Noto Sans&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;, &quot;Segoe UI Symbol&quot;, &quot;Noto Color Emoji&quot;; font-size: 14.7px; font-style: normal; font-weight: 400;">7.906069755554199</span></td>
66
- </tr>
67
  </tbody>
68
  </table>
69
  </body>
@@ -135,7 +135,7 @@ The intended use-case for this model is fictional writing for entertainment purp
135
  As such, it was **not** fine-tuned to be safe and harmless: the base model _and_ this fine-tune have been trained on data known to contain profanity and texts that are lewd or otherwise offensive. It may produce socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive. Outputs might often be factually wrong or misleading.
136
 
137
  <hr>
138
- <p><strong><font size="5">Benchmarks of different quantized variations</font></strong></p>
139
  <html>
140
  <head>
141
  <style>
@@ -160,89 +160,1673 @@ As such, it was **not** fine-tuned to be safe and harmless: the base model _and_
160
  </style>
161
  </head>
162
  <body>
163
- <table>
164
- <thead>
165
- <tr>
166
- <th>GPTQ Variation:</th>
167
- <th>Wikitext2</th>
168
- <th>Ptb-New</th>
169
- <th>C4-New</th>
170
- </tr>
171
- </thead>
172
- <tbody>
173
- <tr>
174
- <td>Metharme 7b - 16bit</td>
175
- <td>5.7208476066589355</td>
176
- <td>41.61103439331055</td>
177
- <td>7.512405872344971</td>
178
- </tr>
179
- <tr>
180
- <td>4bit --act-order</td>
181
- <td>6.281311511993408</td>
182
- <td>46.79158401489258</td>
183
- <td>7.906069755554199</td>
184
- </tr>
185
- <tr>
186
- <td>4bit --act-order --sym</td>
187
- <td>6.477842807769775</td>
188
- <td>48.53507614135742</td>
189
- <td>7.993765354156494</td>
190
- </tr>
191
- <tr>
192
- <td>4bit alone</td>
193
- <td>6.997134685516357</td>
194
- <td>58.87525177001953</td>
195
- <td>8.824191093444824</td>
196
- </tr>
197
- <tr>
198
- <td>4bit --trits</td>
199
- <td>7.832409858703613</td>
200
- <td>55.383026123046875</td>
201
- <td>8.806737899780273</td>
202
- </tr>
203
- <tr>
204
- <td>4bit --groupsize 32</td>
205
- <td>6.902080535888672</td>
206
- <td>50.237754821777344</td>
207
- <td>8.081602096557617</td>
208
- </tr>
209
- <tr>
210
- <td>4bit --groupsize 128</td>
211
- <td>10.206376075744629</td>
212
- <td>49.00401306152344</td>
213
- <td>8.198845863342285</td>
214
- </tr>
215
- <tr>
216
- <td>4bit --true-sequential</td>
217
- <td>7.735969543457031</td>
218
- <td>62.92238235473633</td>
219
- <td>8.913650512695312</td>
220
- </tr>
221
- <tr>
222
- <td>4bit --true-sequential --groupsize 32</td>
223
- <td>6.424218654632568</td>
224
- <td>48.48588943481445</td>
225
- <td>8.089512825012207</td>
226
- </tr>
227
- <tr>
228
- <td>4bit --true-sequential --groupsize 128</td>
229
- <td>9.475017547607422</td>
230
- <td>52.358829498291016</td>
231
- <td>8.340700149536133</td>
232
- </tr>
233
- <tr>
234
- <td>4bit --act-order --groupsize 32 - ERROR</td>
235
- <td>275425.5</td>
236
- <td>267733.25</td>
237
- <td>254506.71875</td>
238
- </tr>
239
- <tr>
240
- <td>4bit --act-order --groupsize 128 - ERROR</td>
241
- <td>251321.265625</td>
242
- <td>250117.859375</td>
243
- <td>232929.234375</td>
244
- </tr>
245
- </tbody>
246
- </table>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
247
  </body>
248
  </html>
 
17
 
18
  Quantized for KoboldAI use using https://github.com/0cc4m/GPTQ-for-LLaMa
19
 
20
+ I created several dozen quantized variations of this model and believe this variation to be "best." <br>
21
  <html>
22
  <head>
23
  <style>
 
46
  <thead>
47
  <tr>
48
  <th>Model:</th>
49
+ <th>Wikitext2</th>
50
  <th>Ptb-New</th>
51
  <th>C4-New</th>
52
  </tr>
 
58
  <td>41.61103439331055</td>
59
  <td>7.512405872344971</td>
60
  </tr>
61
+ <tr>
62
+ <td>Metharme-AO-TS-Trits-damp0.1</td>
63
+ <td>6.172733783721924</td>
64
+ <td>45.19890213012695</td>
65
+ <td>7.872506141662598</td>
66
+ </tr>
67
  </tbody>
68
  </table>
69
  </body>
 
135
  As such, it was **not** fine-tuned to be safe and harmless: the base model _and_ this fine-tune have been trained on data known to contain profanity and texts that are lewd or otherwise offensive. It may produce socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive. Outputs might often be factually wrong or misleading.
136
 
137
  <hr>
138
+ <p><strong><font size="5">Click to Expand Benchmarks of different quantized variations</font></strong></p>
139
  <html>
140
  <head>
141
  <style>
 
160
  </style>
161
  </head>
162
  <body>
163
+ <details>
164
+ <summary>Benchmarks Sorted by C4-New score</summary>
165
+ <table>
166
+ <thead>
167
+ <tr>
168
+ <th>GPTQ Variation:</th>
169
+ <th>Wikitext2</th>
170
+ <th>Ptb-New</th>
171
+ <th>C4-New</th>
172
+ </tr>
173
+ </thead>
174
+ <tbody>
175
+ <tr>
176
+ <td>Metharme-7b-16bit</td>
177
+ <td>5.7208476066589355</td>
178
+ <td>41.61103439331055</td>
179
+ <td>7.512405872344971</td>
180
+ </tr>
181
+ <tr>
182
+ <td>Metharme-ao-ts-trits-damp0.1</td>
183
+ <td>6.172733783721924</td>
184
+ <td>45.19890213012695</td>
185
+ <td>7.872506141662598</td>
186
+ </tr>
187
+ <tr>
188
+ <td>Metharme-ao-trits-damp0.1</td>
189
+ <td>6.163661956787109</td>
190
+ <td>46.50249099731445</td>
191
+ <td>7.877425193786621</td>
192
+ </tr>
193
+ <tr>
194
+ <td>Metharme-ao-ts-damp0.1</td>
195
+ <td>6.184001445770264</td>
196
+ <td>46.17180633544922</td>
197
+ <td>7.880400657653809</td>
198
+ </tr>
199
+ <tr>
200
+ <td>Metharme-7b-4bit-ao-damp0.1</td>
201
+ <td>6.220707893371582</td>
202
+ <td>47.82929611206055</td>
203
+ <td>7.884565353393555</td>
204
+ </tr>
205
+ <tr>
206
+ <td>Metharme-ao-ts-trits</td>
207
+ <td>6.310682773590088</td>
208
+ <td>46.4483757019043</td>
209
+ <td>7.898126602172852</td>
210
+ </tr>
211
+ <tr>
212
+ <td>Metharme-7b-4bit-ao</td>
213
+ <td>6.281311511993408</td>
214
+ <td>46.79158401489258</td>
215
+ <td>7.906069755554199</td>
216
+ </tr>
217
+ <tr>
218
+ <td>Metharme-ao-trits</td>
219
+ <td>6.283935546875</td>
220
+ <td>46.57590103149414</td>
221
+ <td>7.907411575317383</td>
222
+ </tr>
223
+ <tr>
224
+ <td>Metharme-ao-ts</td>
225
+ <td>6.329496383666992</td>
226
+ <td>46.88129806518555</td>
227
+ <td>7.910323143005371</td>
228
+ </tr>
229
+ <tr>
230
+ <td>Metharme-ao-ts-sym-trits-damp0.1</td>
231
+ <td>6.232576370239258</td>
232
+ <td>48.081459045410156</td>
233
+ <td>7.95023250579834</td>
234
+ </tr>
235
+ <tr>
236
+ <td>Metharme-ao-ts-sym-damp0.1</td>
237
+ <td>6.210323333740234</td>
238
+ <td>47.66789245605469</td>
239
+ <td>7.952476978302002</td>
240
+ </tr>
241
+ <tr>
242
+ <td>Metharme-ao-sym-trits-damp0.1</td>
243
+ <td>6.329384803771973</td>
244
+ <td>48.06882858276367</td>
245
+ <td>7.959168910980225</td>
246
+ </tr>
247
+ <tr>
248
+ <td>Metharme-ao-ts-sym-trits</td>
249
+ <td>6.471063137054443</td>
250
+ <td>49.650611877441406</td>
251
+ <td>7.969552040100098</td>
252
+ </tr>
253
+ <tr>
254
+ <td>Metharme-ao-ts-sym</td>
255
+ <td>6.460526943206787</td>
256
+ <td>47.190460205078125</td>
257
+ <td>7.9732160568237305</td>
258
+ </tr>
259
+ <tr>
260
+ <td>Metharme-ao-sym-trits</td>
261
+ <td>6.390106678009033</td>
262
+ <td>48.15375900268555</td>
263
+ <td>7.9804582595825195</td>
264
+ </tr>
265
+ <tr>
266
+ <td>Metharme-7b-4bit-ao-sym</td>
267
+ <td>6.477842807769775</td>
268
+ <td>48.53507614135742</td>
269
+ <td>7.993765354156494</td>
270
+ </tr>
271
+ <tr>
272
+ <td>Metharme-ao-sym</td>
273
+ <td>6.477842807769775</td>
274
+ <td>48.53507614135742</td>
275
+ <td>7.993765354156494</td>
276
+ </tr>
277
+ <tr>
278
+ <td>Metharme-7b-4bit-ts-32g-trits</td>
279
+ <td>6.632943153381348</td>
280
+ <td>47.973228454589844</td>
281
+ <td>8.013848304748535</td>
282
+ </tr>
283
+ <tr>
284
+ <td>Metharme-7b-4bit-ts-32g-sym-damp0.1</td>
285
+ <td>6.274552822113037</td>
286
+ <td>47.35737228393555</td>
287
+ <td>8.06270980834961</td>
288
+ </tr>
289
+ <tr>
290
+ <td>Metharme-7b-4bit-ts-32g-sym-trits-damp0.1</td>
291
+ <td>6.266031265258789</td>
292
+ <td>47.346702575683594</td>
293
+ <td>8.068148612976074</td>
294
+ </tr>
295
+ <tr>
296
+ <td>Metharme-7b-4bit-32g-damp0.1</td>
297
+ <td>6.107605934143066</td>
298
+ <td>47.91380310058594</td>
299
+ <td>8.068695068359375</td>
300
+ </tr>
301
+ <tr>
302
+ <td>Metharme-7b-4bit-32g-trits-damp0.1</td>
303
+ <td>6.128157138824463</td>
304
+ <td>48.04175567626953</td>
305
+ <td>8.0708646774292</td>
306
+ </tr>
307
+ <tr>
308
+ <td>Metharme-7b-4bit-ts-32g-damp0.1</td>
309
+ <td>6.219024658203125</td>
310
+ <td>45.834869384765625</td>
311
+ <td>8.071272850036621</td>
312
+ </tr>
313
+ <tr>
314
+ <td>Metharme-7b-4bit-32g-trits</td>
315
+ <td>7.017086029052734</td>
316
+ <td>45.04129409790039</td>
317
+ <td>8.074845314025879</td>
318
+ </tr>
319
+ <tr>
320
+ <td>Metharme-7b-4bit-32g-sym-trits-damp0.1</td>
321
+ <td>6.109438896179199</td>
322
+ <td>47.35737228393555</td>
323
+ <td>8.075060844421387</td>
324
+ </tr>
325
+ <tr>
326
+ <td>Metharme-7b-4bit-ts-32g-trits-damp0.1</td>
327
+ <td>6.118431568145752</td>
328
+ <td>45.67333221435547</td>
329
+ <td>8.077078819274902</td>
330
+ </tr>
331
+ <tr>
332
+ <td>Metharme-7b-4bit-32g</td>
333
+ <td>6.902080535888672</td>
334
+ <td>50.237754821777344</td>
335
+ <td>8.081602096557617</td>
336
+ </tr>
337
+ <tr>
338
+ <td>Metharme-7b-4bit-ts-32g</td>
339
+ <td>6.424218654632568</td>
340
+ <td>48.48588943481445</td>
341
+ <td>8.089512825012207</td>
342
+ </tr>
343
+ <tr>
344
+ <td>Metharme-7b-4bit-ts-32g-sym-trits</td>
345
+ <td>6.82415771484375</td>
346
+ <td>48.82029724121094</td>
347
+ <td>8.090987205505371</td>
348
+ </tr>
349
+ <tr>
350
+ <td>Metharme-7b-4bit-32g-sym-damp0.1</td>
351
+ <td>6.566899299621582</td>
352
+ <td>48.0670166015625</td>
353
+ <td>8.095841407775879</td>
354
+ </tr>
355
+ <tr>
356
+ <td>Metharme-7b-4bit-ts-128g-trits-damp0.1</td>
357
+ <td>6.289113521575928</td>
358
+ <td>46.06787109375</td>
359
+ <td>8.122251510620117</td>
360
+ </tr>
361
+ <tr>
362
+ <td>Metharme-7b-4bit-32g-sym</td>
363
+ <td>6.518134117126465</td>
364
+ <td>49.66925811767578</td>
365
+ <td>8.13516616821289</td>
366
+ </tr>
367
+ <tr>
368
+ <td>Metharme-7b-4bit-32g-sym-trits</td>
369
+ <td>6.206963539123535</td>
370
+ <td>46.88833999633789</td>
371
+ <td>8.13610553741455</td>
372
+ </tr>
373
+ <tr>
374
+ <td>Metharme-7b-4bit-128g-damp0.1</td>
375
+ <td>6.242006301879883</td>
376
+ <td>45.30938720703125</td>
377
+ <td>8.14249324798584</td>
378
+ </tr>
379
+ <tr>
380
+ <td>Metharme-7b-4bit-ts-32g-sym</td>
381
+ <td>6.387663841247559</td>
382
+ <td>48.07244110107422</td>
383
+ <td>8.173730850219727</td>
384
+ </tr>
385
+ <tr>
386
+ <td>Metharme-7b-4bit-128g-trits-damp0.1</td>
387
+ <td>6.262309551239014</td>
388
+ <td>47.80055618286133</td>
389
+ <td>8.192194938659668</td>
390
+ </tr>
391
+ <tr>
392
+ <td>Metharme-7b-4bit-128g</td>
393
+ <td>10.206376075744629</td>
394
+ <td>49.00401306152344</td>
395
+ <td>8.198845863342285</td>
396
+ </tr>
397
+ <tr>
398
+ <td>Metharme-7b-4bit-ts-128g-damp0.1</td>
399
+ <td>6.17774772644043</td>
400
+ <td>46.47630310058594</td>
401
+ <td>8.20170783996582</td>
402
+ </tr>
403
+ <tr>
404
+ <td>Metharme-7b-4bit-ts-128g-sym-trits-damp0.1</td>
405
+ <td>6.225503921508789</td>
406
+ <td>53.12746047973633</td>
407
+ <td>8.240595817565918</td>
408
+ </tr>
409
+ <tr>
410
+ <td>Metharme-7b-4bit-128g-trits</td>
411
+ <td>8.68796443939209</td>
412
+ <td>49.73833465576172</td>
413
+ <td>8.2406587600708</td>
414
+ </tr>
415
+ <tr>
416
+ <td>Metharme-7b-4bit-ts-128g-sym-damp0.1</td>
417
+ <td>6.584965705871582</td>
418
+ <td>55.20026397705078</td>
419
+ <td>8.268644332885742</td>
420
+ </tr>
421
+ <tr>
422
+ <td>Metharme-7b-4bit-ts-128g-trits</td>
423
+ <td>7.350858688354492</td>
424
+ <td>44.25314712524414</td>
425
+ <td>8.274221420288086</td>
426
+ </tr>
427
+ <tr>
428
+ <td>Metharme-7b-4bit-128g-sym-trits-damp0.1</td>
429
+ <td>6.585468769073486</td>
430
+ <td>51.55869674682617</td>
431
+ <td>8.2803316116333</td>
432
+ </tr>
433
+ <tr>
434
+ <td>Metharme-7b-4bit-ts-128g-sym-trits</td>
435
+ <td>6.756448745727539</td>
436
+ <td>51.510311126708984</td>
437
+ <td>8.292160987854004</td>
438
+ </tr>
439
+ <tr>
440
+ <td>Metharme-7b-4bit-128g-sym-damp0.1</td>
441
+ <td>6.379064083099365</td>
442
+ <td>52.17233657836914</td>
443
+ <td>8.316649436950684</td>
444
+ </tr>
445
+ <tr>
446
+ <td>Metharme-7b-4bit-128g-sym-trits</td>
447
+ <td>7.056288242340088</td>
448
+ <td>48.983768463134766</td>
449
+ <td>8.339276313781738</td>
450
+ </tr>
451
+ <tr>
452
+ <td>Metharme-7b-4bit-ts-128g</td>
453
+ <td>9.475017547607422</td>
454
+ <td>52.358829498291016</td>
455
+ <td>8.340700149536133</td>
456
+ </tr>
457
+ <tr>
458
+ <td>Metharme-7b-4bit-128g-sym</td>
459
+ <td>6.9575653076171875</td>
460
+ <td>49.356834411621094</td>
461
+ <td>8.35644817352295</td>
462
+ </tr>
463
+ <tr>
464
+ <td>Metharme-7b-4bit-ts-128g-sym</td>
465
+ <td>6.819341659545898</td>
466
+ <td>55.28740310668945</td>
467
+ <td>8.377721786499023</td>
468
+ </tr>
469
+ <tr>
470
+ <td>Metharme-7b-4bit-ts-damp0.1</td>
471
+ <td>6.7783050537109375</td>
472
+ <td>51.81301498413086</td>
473
+ <td>8.621373176574707</td>
474
+ </tr>
475
+ <tr>
476
+ <td>Metharme-7b-4bit-ts-trits-damp0.1</td>
477
+ <td>6.631694793701172</td>
478
+ <td>51.7371711730957</td>
479
+ <td>8.656966209411621</td>
480
+ </tr>
481
+ <tr>
482
+ <td>Metharme-7b-4bit-damp0.1</td>
483
+ <td>6.495014190673828</td>
484
+ <td>49.39763641357422</td>
485
+ <td>8.68167781829834</td>
486
+ </tr>
487
+ <tr>
488
+ <td>Metharme-sym-damp0.1</td>
489
+ <td>6.896804332733154</td>
490
+ <td>57.4250602722168</td>
491
+ <td>8.703770637512207</td>
492
+ </tr>
493
+ <tr>
494
+ <td>Metharme-7b-4bit-ts-sym</td>
495
+ <td>7.270263671875</td>
496
+ <td>54.35262680053711</td>
497
+ <td>8.787986755371094</td>
498
+ </tr>
499
+ <tr>
500
+ <td>Metharme-7b-4bit-trits</td>
501
+ <td>7.832409858703613</td>
502
+ <td>55.383026123046875</td>
503
+ <td>8.806737899780273</td>
504
+ </tr>
505
+ <tr>
506
+ <td>Metharme-trits</td>
507
+ <td>7.832409858703613</td>
508
+ <td>55.383026123046875</td>
509
+ <td>8.806737899780273</td>
510
+ </tr>
511
+ <tr>
512
+ <td>Metharme-7b-4bit-ts-sym-damp0.1</td>
513
+ <td>6.7517595291137695</td>
514
+ <td>54.06147384643555</td>
515
+ <td>8.821818351745605</td>
516
+ </tr>
517
+ <tr>
518
+ <td>Metharme-7b-4bit-alone</td>
519
+ <td>6.997134685516357</td>
520
+ <td>58.87525177001953</td>
521
+ <td>8.824191093444824</td>
522
+ </tr>
523
+ <tr>
524
+ <td>Metharme-7b-4bit-ts-trits</td>
525
+ <td>7.2306809425354</td>
526
+ <td>66.78710174560547</td>
527
+ <td>8.879831314086914</td>
528
+ </tr>
529
+ <tr>
530
+ <td>Metharme-7b-4bit-ts-sym-trits-damp0.1</td>
531
+ <td>6.886506080627441</td>
532
+ <td>64.72743225097656</td>
533
+ <td>8.880627632141113</td>
534
+ </tr>
535
+ <tr>
536
+ <td>Metharme-7b-4bit-ts</td>
537
+ <td>7.735969543457031</td>
538
+ <td>62.92238235473633</td>
539
+ <td>8.913650512695312</td>
540
+ </tr>
541
+ <tr>
542
+ <td>Metharme-sym-trits-damp0.1</td>
543
+ <td>7.075908184051514</td>
544
+ <td>59.13897705078125</td>
545
+ <td>8.919178009033203</td>
546
+ </tr>
547
+ <tr>
548
+ <td>Metharme-7b-4bit-ts-sym-trits</td>
549
+ <td>7.599876403808594</td>
550
+ <td>55.75454330444336</td>
551
+ <td>8.932201385498047</td>
552
+ </tr>
553
+ <tr>
554
+ <td>Metharme-sym-trits</td>
555
+ <td>7.494253635406494</td>
556
+ <td>63.320709228515625</td>
557
+ <td>8.969240188598633</td>
558
+ </tr>
559
+ <tr>
560
+ <td>Metharme-sym</td>
561
+ <td>7.585672855377197</td>
562
+ <td>61.01168441772461</td>
563
+ <td>9.032520294189453</td>
564
+ </tr>
565
+ <tr>
566
+ <td>Metharme-7b-4bit-ao-128g</td>
567
+ <td>251321.265625</td>
568
+ <td>250117.859375</td>
569
+ <td>232929.234375</td>
570
+ </tr>
571
+ <tr>
572
+ <td>Metharme-7b-4bit-ao-32g</td>
573
+ <td>275425.5</td>
574
+ <td>267733.25</td>
575
+ <td>254506.71875</td>
576
+ </tr>
577
+ </tbody>
578
+ </table>
579
+ </details>
580
+ <details>
581
+ <summary>Benchmarks Sorted by Wikitext2</summary>
582
+ <table>
583
+ <thead>
584
+ <tr>
585
+ <th>GPTQ Variation:</th>
586
+ <th>Wikitext2</th>
587
+ <th>Ptb-New</th>
588
+ <th>C4-New</th>
589
+ </tr>
590
+ </thead>
591
+ <tbody>
592
+ <tr>
593
+ <td>Metharme-7b-16bit</td>
594
+ <td>5.7208476066589355</td>
595
+ <td>41.61103439331055</td>
596
+ <td>7.512405872344971</td>
597
+ </tr>
598
+ <tr>
599
+ <td>Metharme-7b-4bit-32g-damp0.1</td>
600
+ <td>6.107605934143066</td>
601
+ <td>47.91380310058594</td>
602
+ <td>8.068695068359375</td>
603
+ </tr>
604
+ <tr>
605
+ <td>Metharme-7b-4bit-32g-sym-trits-damp0.1</td>
606
+ <td>6.109438896179199</td>
607
+ <td>47.35737228393555</td>
608
+ <td>8.075060844421387</td>
609
+ </tr>
610
+ <tr>
611
+ <td>Metharme-7b-4bit-ts-32g-trits-damp0.1</td>
612
+ <td>6.118431568145752</td>
613
+ <td>45.67333221435547</td>
614
+ <td>8.077078819274902</td>
615
+ </tr>
616
+ <tr>
617
+ <td>Metharme-7b-4bit-32g-trits-damp0.1</td>
618
+ <td>6.128157138824463</td>
619
+ <td>48.04175567626953</td>
620
+ <td>8.0708646774292</td>
621
+ </tr>
622
+ <tr>
623
+ <td>Metharme-ao-trits-damp0.1</td>
624
+ <td>6.163661956787109</td>
625
+ <td>46.50249099731445</td>
626
+ <td>7.877425193786621</td>
627
+ </tr>
628
+ <tr>
629
+ <td>Metharme-ao-ts-trits-damp0.1</td>
630
+ <td>6.172733783721924</td>
631
+ <td>45.19890213012695</td>
632
+ <td>7.872506141662598</td>
633
+ </tr>
634
+ <tr>
635
+ <td>Metharme-7b-4bit-ts-128g-damp0.1</td>
636
+ <td>6.17774772644043</td>
637
+ <td>46.47630310058594</td>
638
+ <td>8.20170783996582</td>
639
+ </tr>
640
+ <tr>
641
+ <td>Metharme-ao-ts-damp0.1</td>
642
+ <td>6.184001445770264</td>
643
+ <td>46.17180633544922</td>
644
+ <td>7.880400657653809</td>
645
+ </tr>
646
+ <tr>
647
+ <td>Metharme-7b-4bit-32g-sym-trits</td>
648
+ <td>6.206963539123535</td>
649
+ <td>46.88833999633789</td>
650
+ <td>8.13610553741455</td>
651
+ </tr>
652
+ <tr>
653
+ <td>Metharme-ao-ts-sym-damp0.1</td>
654
+ <td>6.210323333740234</td>
655
+ <td>47.66789245605469</td>
656
+ <td>7.952476978302002</td>
657
+ </tr>
658
+ <tr>
659
+ <td>Metharme-7b-4bit-ts-32g-damp0.1</td>
660
+ <td>6.219024658203125</td>
661
+ <td>45.834869384765625</td>
662
+ <td>8.071272850036621</td>
663
+ </tr>
664
+ <tr>
665
+ <td>Metharme-7b-4bit-ao-damp0.1</td>
666
+ <td>6.220707893371582</td>
667
+ <td>47.82929611206055</td>
668
+ <td>7.884565353393555</td>
669
+ </tr>
670
+ <tr>
671
+ <td>Metharme-7b-4bit-ts-128g-sym-trits-damp0.1</td>
672
+ <td>6.225503921508789</td>
673
+ <td>53.12746047973633</td>
674
+ <td>8.240595817565918</td>
675
+ </tr>
676
+ <tr>
677
+ <td>Metharme-ao-ts-sym-trits-damp0.1</td>
678
+ <td>6.232576370239258</td>
679
+ <td>48.081459045410156</td>
680
+ <td>7.95023250579834</td>
681
+ </tr>
682
+ <tr>
683
+ <td>Metharme-7b-4bit-128g-damp0.1</td>
684
+ <td>6.242006301879883</td>
685
+ <td>45.30938720703125</td>
686
+ <td>8.14249324798584</td>
687
+ </tr>
688
+ <tr>
689
+ <td>Metharme-7b-4bit-128g-trits-damp0.1</td>
690
+ <td>6.262309551239014</td>
691
+ <td>47.80055618286133</td>
692
+ <td>8.192194938659668</td>
693
+ </tr>
694
+ <tr>
695
+ <td>Metharme-7b-4bit-ts-32g-sym-trits-damp0.1</td>
696
+ <td>6.266031265258789</td>
697
+ <td>47.346702575683594</td>
698
+ <td>8.068148612976074</td>
699
+ </tr>
700
+ <tr>
701
+ <td>Metharme-7b-4bit-ts-32g-sym-damp0.1</td>
702
+ <td>6.274552822113037</td>
703
+ <td>47.35737228393555</td>
704
+ <td>8.06270980834961</td>
705
+ </tr>
706
+ <tr>
707
+ <td>Metharme-7b-4bit-ao</td>
708
+ <td>6.281311511993408</td>
709
+ <td>46.79158401489258</td>
710
+ <td>7.906069755554199</td>
711
+ </tr>
712
+ <tr>
713
+ <td>Metharme-ao-trits</td>
714
+ <td>6.283935546875</td>
715
+ <td>46.57590103149414</td>
716
+ <td>7.907411575317383</td>
717
+ </tr>
718
+ <tr>
719
+ <td>Metharme-7b-4bit-ts-128g-trits-damp0.1</td>
720
+ <td>6.289113521575928</td>
721
+ <td>46.06787109375</td>
722
+ <td>8.122251510620117</td>
723
+ </tr>
724
+ <tr>
725
+ <td>Metharme-ao-ts-trits</td>
726
+ <td>6.310682773590088</td>
727
+ <td>46.4483757019043</td>
728
+ <td>7.898126602172852</td>
729
+ </tr>
730
+ <tr>
731
+ <td>Metharme-ao-sym-trits-damp0.1</td>
732
+ <td>6.329384803771973</td>
733
+ <td>48.06882858276367</td>
734
+ <td>7.959168910980225</td>
735
+ </tr>
736
+ <tr>
737
+ <td>Metharme-ao-ts</td>
738
+ <td>6.329496383666992</td>
739
+ <td>46.88129806518555</td>
740
+ <td>7.910323143005371</td>
741
+ </tr>
742
+ <tr>
743
+ <td>Metharme-7b-4bit-128g-sym-damp0.1</td>
744
+ <td>6.379064083099365</td>
745
+ <td>52.17233657836914</td>
746
+ <td>8.316649436950684</td>
747
+ </tr>
748
+ <tr>
749
+ <td>Metharme-7b-4bit-ts-32g-sym</td>
750
+ <td>6.387663841247559</td>
751
+ <td>48.07244110107422</td>
752
+ <td>8.173730850219727</td>
753
+ </tr>
754
+ <tr>
755
+ <td>Metharme-ao-sym-trits</td>
756
+ <td>6.390106678009033</td>
757
+ <td>48.15375900268555</td>
758
+ <td>7.9804582595825195</td>
759
+ </tr>
760
+ <tr>
761
+ <td>Metharme-7b-4bit-ts-32g</td>
762
+ <td>6.424218654632568</td>
763
+ <td>48.48588943481445</td>
764
+ <td>8.089512825012207</td>
765
+ </tr>
766
+ <tr>
767
+ <td>Metharme-ao-ts-sym</td>
768
+ <td>6.460526943206787</td>
769
+ <td>47.190460205078125</td>
770
+ <td>7.9732160568237305</td>
771
+ </tr>
772
+ <tr>
773
+ <td>Metharme-ao-ts-sym-trits</td>
774
+ <td>6.471063137054443</td>
775
+ <td>49.650611877441406</td>
776
+ <td>7.969552040100098</td>
777
+ </tr>
778
+ <tr>
779
+ <td>Metharme-7b-4bit-ao-sym</td>
780
+ <td>6.477842807769775</td>
781
+ <td>48.53507614135742</td>
782
+ <td>7.993765354156494</td>
783
+ </tr>
784
+ <tr>
785
+ <td>Metharme-ao-sym</td>
786
+ <td>6.477842807769775</td>
787
+ <td>48.53507614135742</td>
788
+ <td>7.993765354156494</td>
789
+ </tr>
790
+ <tr>
791
+ <td>Metharme-7b-4bit-damp0.1</td>
792
+ <td>6.495014190673828</td>
793
+ <td>49.39763641357422</td>
794
+ <td>8.68167781829834</td>
795
+ </tr>
796
+ <tr>
797
+ <td>Metharme-7b-4bit-32g-sym</td>
798
+ <td>6.518134117126465</td>
799
+ <td>49.66925811767578</td>
800
+ <td>8.13516616821289</td>
801
+ </tr>
802
+ <tr>
803
+ <td>Metharme-7b-4bit-32g-sym-damp0.1</td>
804
+ <td>6.566899299621582</td>
805
+ <td>48.0670166015625</td>
806
+ <td>8.095841407775879</td>
807
+ </tr>
808
+ <tr>
809
+ <td>Metharme-7b-4bit-ts-128g-sym-damp0.1</td>
810
+ <td>6.584965705871582</td>
811
+ <td>55.20026397705078</td>
812
+ <td>8.268644332885742</td>
813
+ </tr>
814
+ <tr>
815
+ <td>Metharme-7b-4bit-128g-sym-trits-damp0.1</td>
816
+ <td>6.585468769073486</td>
817
+ <td>51.55869674682617</td>
818
+ <td>8.2803316116333</td>
819
+ </tr>
820
+ <tr>
821
+ <td>Metharme-7b-4bit-ts-trits-damp0.1</td>
822
+ <td>6.631694793701172</td>
823
+ <td>51.7371711730957</td>
824
+ <td>8.656966209411621</td>
825
+ </tr>
826
+ <tr>
827
+ <td>Metharme-7b-4bit-ts-32g-trits</td>
828
+ <td>6.632943153381348</td>
829
+ <td>47.973228454589844</td>
830
+ <td>8.013848304748535</td>
831
+ </tr>
832
+ <tr>
833
+ <td>Metharme-7b-4bit-ts-sym-damp0.1</td>
834
+ <td>6.7517595291137695</td>
835
+ <td>54.06147384643555</td>
836
+ <td>8.821818351745605</td>
837
+ </tr>
838
+ <tr>
839
+ <td>Metharme-7b-4bit-ts-128g-sym-trits</td>
840
+ <td>6.756448745727539</td>
841
+ <td>51.510311126708984</td>
842
+ <td>8.292160987854004</td>
843
+ </tr>
844
+ <tr>
845
+ <td>Metharme-7b-4bit-ts-damp0.1</td>
846
+ <td>6.7783050537109375</td>
847
+ <td>51.81301498413086</td>
848
+ <td>8.621373176574707</td>
849
+ </tr>
850
+ <tr>
851
+ <td>Metharme-7b-4bit-ts-128g-sym</td>
852
+ <td>6.819341659545898</td>
853
+ <td>55.28740310668945</td>
854
+ <td>8.377721786499023</td>
855
+ </tr>
856
+ <tr>
857
+ <td>Metharme-7b-4bit-ts-32g-sym-trits</td>
858
+ <td>6.82415771484375</td>
859
+ <td>48.82029724121094</td>
860
+ <td>8.090987205505371</td>
861
+ </tr>
862
+ <tr>
863
+ <td>Metharme-7b-4bit-ts-sym-trits-damp0.1</td>
864
+ <td>6.886506080627441</td>
865
+ <td>64.72743225097656</td>
866
+ <td>8.880627632141113</td>
867
+ </tr>
868
+ <tr>
869
+ <td>Metharme-sym-damp0.1</td>
870
+ <td>6.896804332733154</td>
871
+ <td>57.4250602722168</td>
872
+ <td>8.703770637512207</td>
873
+ </tr>
874
+ <tr>
875
+ <td>Metharme-7b-4bit-32g</td>
876
+ <td>6.902080535888672</td>
877
+ <td>50.237754821777344</td>
878
+ <td>8.081602096557617</td>
879
+ </tr>
880
+ <tr>
881
+ <td>Metharme-7b-4bit-128g-sym</td>
882
+ <td>6.9575653076171875</td>
883
+ <td>49.356834411621094</td>
884
+ <td>8.35644817352295</td>
885
+ </tr>
886
+ <tr>
887
+ <td>Metharme-7b-4bit-alone</td>
888
+ <td>6.997134685516357</td>
889
+ <td>58.87525177001953</td>
890
+ <td>8.824191093444824</td>
891
+ </tr>
892
+ <tr>
893
+ <td>Metharme-7b-4bit-32g-trits</td>
894
+ <td>7.017086029052734</td>
895
+ <td>45.04129409790039</td>
896
+ <td>8.074845314025879</td>
897
+ </tr>
898
+ <tr>
899
+ <td>Metharme-7b-4bit-128g-sym-trits</td>
900
+ <td>7.056288242340088</td>
901
+ <td>48.983768463134766</td>
902
+ <td>8.339276313781738</td>
903
+ </tr>
904
+ <tr>
905
+ <td>Metharme-sym-trits-damp0.1</td>
906
+ <td>7.075908184051514</td>
907
+ <td>59.13897705078125</td>
908
+ <td>8.919178009033203</td>
909
+ </tr>
910
+ <tr>
911
+ <td>Metharme-7b-4bit-ts-trits</td>
912
+ <td>7.2306809425354</td>
913
+ <td>66.78710174560547</td>
914
+ <td>8.879831314086914</td>
915
+ </tr>
916
+ <tr>
917
+ <td>Metharme-7b-4bit-ts-sym</td>
918
+ <td>7.270263671875</td>
919
+ <td>54.35262680053711</td>
920
+ <td>8.787986755371094</td>
921
+ </tr>
922
+ <tr>
923
+ <td>Metharme-7b-4bit-ts-128g-trits</td>
924
+ <td>7.350858688354492</td>
925
+ <td>44.25314712524414</td>
926
+ <td>8.274221420288086</td>
927
+ </tr>
928
+ <tr>
929
+ <td>Metharme-sym-trits</td>
930
+ <td>7.494253635406494</td>
931
+ <td>63.320709228515625</td>
932
+ <td>8.969240188598633</td>
933
+ </tr>
934
+ <tr>
935
+ <td>Metharme-sym</td>
936
+ <td>7.585672855377197</td>
937
+ <td>61.01168441772461</td>
938
+ <td>9.032520294189453</td>
939
+ </tr>
940
+ <tr>
941
+ <td>Metharme-7b-4bit-ts-sym-trits</td>
942
+ <td>7.599876403808594</td>
943
+ <td>55.75454330444336</td>
944
+ <td>8.932201385498047</td>
945
+ </tr>
946
+ <tr>
947
+ <td>Metharme-7b-4bit-ts</td>
948
+ <td>7.735969543457031</td>
949
+ <td>62.92238235473633</td>
950
+ <td>8.913650512695312</td>
951
+ </tr>
952
+ <tr>
953
+ <td>Metharme-7b-4bit-trits</td>
954
+ <td>7.832409858703613</td>
955
+ <td>55.383026123046875</td>
956
+ <td>8.806737899780273</td>
957
+ </tr>
958
+ <tr>
959
+ <td>Metharme-trits</td>
960
+ <td>7.832409858703613</td>
961
+ <td>55.383026123046875</td>
962
+ <td>8.806737899780273</td>
963
+ </tr>
964
+ <tr>
965
+ <td>Metharme-7b-4bit-128g-trits</td>
966
+ <td>8.68796443939209</td>
967
+ <td>49.73833465576172</td>
968
+ <td>8.2406587600708</td>
969
+ </tr>
970
+ <tr>
971
+ <td>Metharme-7b-4bit-ts-128g</td>
972
+ <td>9.475017547607422</td>
973
+ <td>52.358829498291016</td>
974
+ <td>8.340700149536133</td>
975
+ </tr>
976
+ <tr>
977
+ <td>Metharme-7b-4bit-128g</td>
978
+ <td>10.206376075744629</td>
979
+ <td>49.00401306152344</td>
980
+ <td>8.198845863342285</td>
981
+ </tr>
982
+ <tr>
983
+ <td>Metharme-7b-4bit-ao-128g</td>
984
+ <td>251321.265625</td>
985
+ <td>250117.859375</td>
986
+ <td>232929.234375</td>
987
+ </tr>
988
+ <tr>
989
+ <td>Metharme-7b-4bit-ao-32g</td>
990
+ <td>275425.5</td>
991
+ <td>267733.25</td>
992
+ <td>254506.71875</td>
993
+ </tr>
994
+ </tbody>
995
+ </table>
996
+ </details>
997
+ <details>
998
+ <summary>Benchmarks Sorted by PTB-new Score</summary>
999
+ <table>
1000
+ <thead>
1001
+ <tr>
1002
+ <th>GPTQ Variation:</th>
1003
+ <th>Wikitext2</th>
1004
+ <th>Ptb-New</th>
1005
+ <th>C4-New</th>
1006
+ </tr>
1007
+ </thead>
1008
+ <tbody>
1009
+ <tr>
1010
+ <td>Metharme-7b-16bit</td>
1011
+ <td>5.7208476066589355</td>
1012
+ <td>41.61103439331055</td>
1013
+ <td>7.512405872344971</td>
1014
+ </tr>
1015
+ <tr>
1016
+ <td>Metharme-7b-4bit-ts-128g-trits</td>
1017
+ <td>7.350858688354492</td>
1018
+ <td>44.25314712524414</td>
1019
+ <td>8.274221420288086</td>
1020
+ </tr>
1021
+ <tr>
1022
+ <td>Metharme-7b-4bit-32g-trits</td>
1023
+ <td>7.017086029052734</td>
1024
+ <td>45.04129409790039</td>
1025
+ <td>8.074845314025879</td>
1026
+ </tr>
1027
+ <tr>
1028
+ <td>Metharme-ao-ts-trits-damp0.1</td>
1029
+ <td>6.172733783721924</td>
1030
+ <td>45.19890213012695</td>
1031
+ <td>7.872506141662598</td>
1032
+ </tr>
1033
+ <tr>
1034
+ <td>Metharme-7b-4bit-128g-damp0.1</td>
1035
+ <td>6.242006301879883</td>
1036
+ <td>45.30938720703125</td>
1037
+ <td>8.14249324798584</td>
1038
+ </tr>
1039
+ <tr>
1040
+ <td>Metharme-7b-4bit-ts-32g-trits-damp0.1</td>
1041
+ <td>6.118431568145752</td>
1042
+ <td>45.67333221435547</td>
1043
+ <td>8.077078819274902</td>
1044
+ </tr>
1045
+ <tr>
1046
+ <td>Metharme-7b-4bit-ts-32g-damp0.1</td>
1047
+ <td>6.219024658203125</td>
1048
+ <td>45.834869384765625</td>
1049
+ <td>8.071272850036621</td>
1050
+ </tr>
1051
+ <tr>
1052
+ <td>Metharme-7b-4bit-ts-128g-trits-damp0.1</td>
1053
+ <td>6.289113521575928</td>
1054
+ <td>46.06787109375</td>
1055
+ <td>8.122251510620117</td>
1056
+ </tr>
1057
+ <tr>
1058
+ <td>Metharme-ao-ts-damp0.1</td>
1059
+ <td>6.184001445770264</td>
1060
+ <td>46.17180633544922</td>
1061
+ <td>7.880400657653809</td>
1062
+ </tr>
1063
+ <tr>
1064
+ <td>Metharme-ao-ts-trits</td>
1065
+ <td>6.310682773590088</td>
1066
+ <td>46.4483757019043</td>
1067
+ <td>7.898126602172852</td>
1068
+ </tr>
1069
+ <tr>
1070
+ <td>Metharme-7b-4bit-ts-128g-damp0.1</td>
1071
+ <td>6.17774772644043</td>
1072
+ <td>46.47630310058594</td>
1073
+ <td>8.20170783996582</td>
1074
+ </tr>
1075
+ <tr>
1076
+ <td>Metharme-ao-trits-damp0.1</td>
1077
+ <td>6.163661956787109</td>
1078
+ <td>46.50249099731445</td>
1079
+ <td>7.877425193786621</td>
1080
+ </tr>
1081
+ <tr>
1082
+ <td>Metharme-ao-trits</td>
1083
+ <td>6.283935546875</td>
1084
+ <td>46.57590103149414</td>
1085
+ <td>7.907411575317383</td>
1086
+ </tr>
1087
+ <tr>
1088
+ <td>Metharme-7b-4bit-ao</td>
1089
+ <td>6.281311511993408</td>
1090
+ <td>46.79158401489258</td>
1091
+ <td>7.906069755554199</td>
1092
+ </tr>
1093
+ <tr>
1094
+ <td>Metharme-ao-ts</td>
1095
+ <td>6.329496383666992</td>
1096
+ <td>46.88129806518555</td>
1097
+ <td>7.910323143005371</td>
1098
+ </tr>
1099
+ <tr>
1100
+ <td>Metharme-7b-4bit-32g-sym-trits</td>
1101
+ <td>6.206963539123535</td>
1102
+ <td>46.88833999633789</td>
1103
+ <td>8.13610553741455</td>
1104
+ </tr>
1105
+ <tr>
1106
+ <td>Metharme-ao-ts-sym</td>
1107
+ <td>6.460526943206787</td>
1108
+ <td>47.190460205078125</td>
1109
+ <td>7.9732160568237305</td>
1110
+ </tr>
1111
+ <tr>
1112
+ <td>Metharme-7b-4bit-ts-32g-sym-trits-damp0.1</td>
1113
+ <td>6.266031265258789</td>
1114
+ <td>47.346702575683594</td>
1115
+ <td>8.068148612976074</td>
1116
+ </tr>
1117
+ <tr>
1118
+ <td>Metharme-7b-4bit-ts-32g-sym-damp0.1</td>
1119
+ <td>6.274552822113037</td>
1120
+ <td>47.35737228393555</td>
1121
+ <td>8.06270980834961</td>
1122
+ </tr>
1123
+ <tr>
1124
+ <td>Metharme-7b-4bit-32g-sym-trits-damp0.1</td>
1125
+ <td>6.109438896179199</td>
1126
+ <td>47.35737228393555</td>
1127
+ <td>8.075060844421387</td>
1128
+ </tr>
1129
+ <tr>
1130
+ <td>Metharme-ao-ts-sym-damp0.1</td>
1131
+ <td>6.210323333740234</td>
1132
+ <td>47.66789245605469</td>
1133
+ <td>7.952476978302002</td>
1134
+ </tr>
1135
+ <tr>
1136
+ <td>Metharme-7b-4bit-128g-trits-damp0.1</td>
1137
+ <td>6.262309551239014</td>
1138
+ <td>47.80055618286133</td>
1139
+ <td>8.192194938659668</td>
1140
+ </tr>
1141
+ <tr>
1142
+ <td>Metharme-7b-4bit-ao-damp0.1</td>
1143
+ <td>6.220707893371582</td>
1144
+ <td>47.82929611206055</td>
1145
+ <td>7.884565353393555</td>
1146
+ </tr>
1147
+ <tr>
1148
+ <td>Metharme-7b-4bit-32g-damp0.1</td>
1149
+ <td>6.107605934143066</td>
1150
+ <td>47.91380310058594</td>
1151
+ <td>8.068695068359375</td>
1152
+ </tr>
1153
+ <tr>
1154
+ <td>Metharme-7b-4bit-ts-32g-trits</td>
1155
+ <td>6.632943153381348</td>
1156
+ <td>47.973228454589844</td>
1157
+ <td>8.013848304748535</td>
1158
+ </tr>
1159
+ <tr>
1160
+ <td>Metharme-7b-4bit-32g-trits-damp0.1</td>
1161
+ <td>6.128157138824463</td>
1162
+ <td>48.04175567626953</td>
1163
+ <td>8.0708646774292</td>
1164
+ </tr>
1165
+ <tr>
1166
+ <td>Metharme-7b-4bit-32g-sym-damp0.1</td>
1167
+ <td>6.566899299621582</td>
1168
+ <td>48.0670166015625</td>
1169
+ <td>8.095841407775879</td>
1170
+ </tr>
1171
+ <tr>
1172
+ <td>Metharme-ao-sym-trits-damp0.1</td>
1173
+ <td>6.329384803771973</td>
1174
+ <td>48.06882858276367</td>
1175
+ <td>7.959168910980225</td>
1176
+ </tr>
1177
+ <tr>
1178
+ <td>Metharme-7b-4bit-ts-32g-sym</td>
1179
+ <td>6.387663841247559</td>
1180
+ <td>48.07244110107422</td>
1181
+ <td>8.173730850219727</td>
1182
+ </tr>
1183
+ <tr>
1184
+ <td>Metharme-ao-ts-sym-trits-damp0.1</td>
1185
+ <td>6.232576370239258</td>
1186
+ <td>48.081459045410156</td>
1187
+ <td>7.95023250579834</td>
1188
+ </tr>
1189
+ <tr>
1190
+ <td>Metharme-ao-sym-trits</td>
1191
+ <td>6.390106678009033</td>
1192
+ <td>48.15375900268555</td>
1193
+ <td>7.9804582595825195</td>
1194
+ </tr>
1195
+ <tr>
1196
+ <td>Metharme-7b-4bit-ts-32g</td>
1197
+ <td>6.424218654632568</td>
1198
+ <td>48.48588943481445</td>
1199
+ <td>8.089512825012207</td>
1200
+ </tr>
1201
+ <tr>
1202
+ <td>Metharme-7b-4bit-ao-sym</td>
1203
+ <td>6.477842807769775</td>
1204
+ <td>48.53507614135742</td>
1205
+ <td>7.993765354156494</td>
1206
+ </tr>
1207
+ <tr>
1208
+ <td>Metharme-ao-sym</td>
1209
+ <td>6.477842807769775</td>
1210
+ <td>48.53507614135742</td>
1211
+ <td>7.993765354156494</td>
1212
+ </tr>
1213
+ <tr>
1214
+ <td>Metharme-7b-4bit-ts-32g-sym-trits</td>
1215
+ <td>6.82415771484375</td>
1216
+ <td>48.82029724121094</td>
1217
+ <td>8.090987205505371</td>
1218
+ </tr>
1219
+ <tr>
1220
+ <td>Metharme-7b-4bit-128g-sym-trits</td>
1221
+ <td>7.056288242340088</td>
1222
+ <td>48.983768463134766</td>
1223
+ <td>8.339276313781738</td>
1224
+ </tr>
1225
+ <tr>
1226
+ <td>Metharme-7b-4bit-128g</td>
1227
+ <td>10.206376075744629</td>
1228
+ <td>49.00401306152344</td>
1229
+ <td>8.198845863342285</td>
1230
+ </tr>
1231
+ <tr>
1232
+ <td>Metharme-7b-4bit-128g-sym</td>
1233
+ <td>6.9575653076171875</td>
1234
+ <td>49.356834411621094</td>
1235
+ <td>8.35644817352295</td>
1236
+ </tr>
1237
+ <tr>
1238
+ <td>Metharme-7b-4bit-damp0.1</td>
1239
+ <td>6.495014190673828</td>
1240
+ <td>49.39763641357422</td>
1241
+ <td>8.68167781829834</td>
1242
+ </tr>
1243
+ <tr>
1244
+ <td>Metharme-ao-ts-sym-trits</td>
1245
+ <td>6.471063137054443</td>
1246
+ <td>49.650611877441406</td>
1247
+ <td>7.969552040100098</td>
1248
+ </tr>
1249
+ <tr>
1250
+ <td>Metharme-7b-4bit-32g-sym</td>
1251
+ <td>6.518134117126465</td>
1252
+ <td>49.66925811767578</td>
1253
+ <td>8.13516616821289</td>
1254
+ </tr>
1255
+ <tr>
1256
+ <td>Metharme-7b-4bit-128g-trits</td>
1257
+ <td>8.68796443939209</td>
1258
+ <td>49.73833465576172</td>
1259
+ <td>8.2406587600708</td>
1260
+ </tr>
1261
+ <tr>
1262
+ <td>Metharme-7b-4bit-32g</td>
1263
+ <td>6.902080535888672</td>
1264
+ <td>50.237754821777344</td>
1265
+ <td>8.081602096557617</td>
1266
+ </tr>
1267
+ <tr>
1268
+ <td>Metharme-7b-4bit-ts-128g-sym-trits</td>
1269
+ <td>6.756448745727539</td>
1270
+ <td>51.510311126708984</td>
1271
+ <td>8.292160987854004</td>
1272
+ </tr>
1273
+ <tr>
1274
+ <td>Metharme-7b-4bit-128g-sym-trits-damp0.1</td>
1275
+ <td>6.585468769073486</td>
1276
+ <td>51.55869674682617</td>
1277
+ <td>8.2803316116333</td>
1278
+ </tr>
1279
+ <tr>
1280
+ <td>Metharme-7b-4bit-ts-trits-damp0.1</td>
1281
+ <td>6.631694793701172</td>
1282
+ <td>51.7371711730957</td>
1283
+ <td>8.656966209411621</td>
1284
+ </tr>
1285
+ <tr>
1286
+ <td>Metharme-7b-4bit-ts-damp0.1</td>
1287
+ <td>6.7783050537109375</td>
1288
+ <td>51.81301498413086</td>
1289
+ <td>8.621373176574707</td>
1290
+ </tr>
1291
+ <tr>
1292
+ <td>Metharme-7b-4bit-128g-sym-damp0.1</td>
1293
+ <td>6.379064083099365</td>
1294
+ <td>52.17233657836914</td>
1295
+ <td>8.316649436950684</td>
1296
+ </tr>
1297
+ <tr>
1298
+ <td>Metharme-7b-4bit-ts-128g</td>
1299
+ <td>9.475017547607422</td>
1300
+ <td>52.358829498291016</td>
1301
+ <td>8.340700149536133</td>
1302
+ </tr>
1303
+ <tr>
1304
+ <td>Metharme-7b-4bit-ts-128g-sym-trits-damp0.1</td>
1305
+ <td>6.225503921508789</td>
1306
+ <td>53.12746047973633</td>
1307
+ <td>8.240595817565918</td>
1308
+ </tr>
1309
+ <tr>
1310
+ <td>Metharme-7b-4bit-ts-sym-damp0.1</td>
1311
+ <td>6.7517595291137695</td>
1312
+ <td>54.06147384643555</td>
1313
+ <td>8.821818351745605</td>
1314
+ </tr>
1315
+ <tr>
1316
+ <td>Metharme-7b-4bit-ts-sym</td>
1317
+ <td>7.270263671875</td>
1318
+ <td>54.35262680053711</td>
1319
+ <td>8.787986755371094</td>
1320
+ </tr>
1321
+ <tr>
1322
+ <td>Metharme-7b-4bit-ts-128g-sym-damp0.1</td>
1323
+ <td>6.584965705871582</td>
1324
+ <td>55.20026397705078</td>
1325
+ <td>8.268644332885742</td>
1326
+ </tr>
1327
+ <tr>
1328
+ <td>Metharme-7b-4bit-ts-128g-sym</td>
1329
+ <td>6.819341659545898</td>
1330
+ <td>55.28740310668945</td>
1331
+ <td>8.377721786499023</td>
1332
+ </tr>
1333
+ <tr>
1334
+ <td>Metharme-7b-4bit-trits</td>
1335
+ <td>7.832409858703613</td>
1336
+ <td>55.383026123046875</td>
1337
+ <td>8.806737899780273</td>
1338
+ </tr>
1339
+ <tr>
1340
+ <td>Metharme-trits</td>
1341
+ <td>7.832409858703613</td>
1342
+ <td>55.383026123046875</td>
1343
+ <td>8.806737899780273</td>
1344
+ </tr>
1345
+ <tr>
1346
+ <td>Metharme-7b-4bit-ts-sym-trits</td>
1347
+ <td>7.599876403808594</td>
1348
+ <td>55.75454330444336</td>
1349
+ <td>8.932201385498047</td>
1350
+ </tr>
1351
+ <tr>
1352
+ <td>Metharme-sym-damp0.1</td>
1353
+ <td>6.896804332733154</td>
1354
+ <td>57.4250602722168</td>
1355
+ <td>8.703770637512207</td>
1356
+ </tr>
1357
+ <tr>
1358
+ <td>Metharme-7b-4bit-alone</td>
1359
+ <td>6.997134685516357</td>
1360
+ <td>58.87525177001953</td>
1361
+ <td>8.824191093444824</td>
1362
+ </tr>
1363
+ <tr>
1364
+ <td>Metharme-sym-trits-damp0.1</td>
1365
+ <td>7.075908184051514</td>
1366
+ <td>59.13897705078125</td>
1367
+ <td>8.919178009033203</td>
1368
+ </tr>
1369
+ <tr>
1370
+ <td>Metharme-sym</td>
1371
+ <td>7.585672855377197</td>
1372
+ <td>61.01168441772461</td>
1373
+ <td>9.032520294189453</td>
1374
+ </tr>
1375
+ <tr>
1376
+ <td>Metharme-7b-4bit-ts</td>
1377
+ <td>7.735969543457031</td>
1378
+ <td>62.92238235473633</td>
1379
+ <td>8.913650512695312</td>
1380
+ </tr>
1381
+ <tr>
1382
+ <td>Metharme-sym-trits</td>
1383
+ <td>7.494253635406494</td>
1384
+ <td>63.320709228515625</td>
1385
+ <td>8.969240188598633</td>
1386
+ </tr>
1387
+ <tr>
1388
+ <td>Metharme-7b-4bit-ts-sym-trits-damp0.1</td>
1389
+ <td>6.886506080627441</td>
1390
+ <td>64.72743225097656</td>
1391
+ <td>8.880627632141113</td>
1392
+ </tr>
1393
+ <tr>
1394
+ <td>Metharme-7b-4bit-ts-trits</td>
1395
+ <td>7.2306809425354</td>
1396
+ <td>66.78710174560547</td>
1397
+ <td>8.879831314086914</td>
1398
+ </tr>
1399
+ <tr>
1400
+ <td>Metharme-7b-4bit-ao-128g</td>
1401
+ <td>251321.265625</td>
1402
+ <td>250117.859375</td>
1403
+ <td>232929.234375</td>
1404
+ </tr>
1405
+ <tr>
1406
+ <td>Metharme-7b-4bit-ao-32g</td>
1407
+ <td>275425.5</td>
1408
+ <td>267733.25</td>
1409
+ <td>254506.71875</td>
1410
+ </tr>
1411
+ </tbody>
1412
+ </table>
1413
+ </details>
1414
+ <details>
1415
+ <summary>Benchmarks Sorted in Alphabetical Order</summary>
1416
+ <table>
1417
+ <thead>
1418
+ <tr>
1419
+ <th>GPTQ Variation:</th>
1420
+ <th>Wikitext2</th>
1421
+ <th>Ptb-New</th>
1422
+ <th>C4-New</th>
1423
+ </tr>
1424
+ </thead>
1425
+ <tbody>
1426
+ <tr>
1427
+ <td>Metharme-7b-16bit</td>
1428
+ <td>5.7208476066589355</td>
1429
+ <td>41.61103439331055</td>
1430
+ <td>7.512405872344971</td>
1431
+ </tr>
1432
+ <tr>
1433
+ <td>Metharme-7b-4bit-128g-damp0.1</td>
1434
+ <td>6.242006301879883</td>
1435
+ <td>45.30938720703125</td>
1436
+ <td>8.14249324798584</td>
1437
+ </tr>
1438
+ <tr>
1439
+ <td>Metharme-7b-4bit-128g-sym-damp0.1</td>
1440
+ <td>6.379064083099365</td>
1441
+ <td>52.17233657836914</td>
1442
+ <td>8.316649436950684</td>
1443
+ </tr>
1444
+ <tr>
1445
+ <td>Metharme-7b-4bit-128g-sym-trits-damp0.1</td>
1446
+ <td>6.585468769073486</td>
1447
+ <td>51.55869674682617</td>
1448
+ <td>8.2803316116333</td>
1449
+ </tr>
1450
+ <tr>
1451
+ <td>Metharme-7b-4bit-128g-trits-damp0.1</td>
1452
+ <td>6.262309551239014</td>
1453
+ <td>47.80055618286133</td>
1454
+ <td>8.192194938659668</td>
1455
+ </tr>
1456
+ <tr>
1457
+ <td>Metharme-7b-4bit-128g</td>
1458
+ <td>10.206376075744629</td>
1459
+ <td>49.00401306152344</td>
1460
+ <td>8.198845863342285</td>
1461
+ </tr>
1462
+ <tr>
1463
+ <td>Metharme-7b-4bit-32g-damp0.1</td>
1464
+ <td>6.107605934143066</td>
1465
+ <td>47.91380310058594</td>
1466
+ <td>8.068695068359375</td>
1467
+ </tr>
1468
+ <tr>
1469
+ <td>Metharme-7b-4bit-32g-sym-damp0.1</td>
1470
+ <td>6.566899299621582</td>
1471
+ <td>48.0670166015625</td>
1472
+ <td>8.095841407775879</td>
1473
+ </tr>
1474
+ <tr>
1475
+ <td>Metharme-7b-4bit-32g-sym-trits-damp0.1</td>
1476
+ <td>6.109438896179199</td>
1477
+ <td>47.35737228393555</td>
1478
+ <td>8.075060844421387</td>
1479
+ </tr>
1480
+ <tr>
1481
+ <td>Metharme-7b-4bit-32g-trits-damp0.1</td>
1482
+ <td>6.128157138824463</td>
1483
+ <td>48.04175567626953</td>
1484
+ <td>8.0708646774292</td>
1485
+ </tr>
1486
+ <tr>
1487
+ <td>Metharme-7b-4bit-32g</td>
1488
+ <td>6.902080535888672</td>
1489
+ <td>50.237754821777344</td>
1490
+ <td>8.081602096557617</td>
1491
+ </tr>
1492
+ <tr>
1493
+ <td>Metharme-7b-4bit-alone</td>
1494
+ <td>6.997134685516357</td>
1495
+ <td>58.87525177001953</td>
1496
+ <td>8.824191093444824</td>
1497
+ </tr>
1498
+ <tr>
1499
+ <td>Metharme-7b-4bit-ao-128g</td>
1500
+ <td>251321.265625</td>
1501
+ <td>250117.859375</td>
1502
+ <td>232929.234375</td>
1503
+ </tr>
1504
+ <tr>
1505
+ <td>Metharme-7b-4bit-ao-32g</td>
1506
+ <td>275425.5</td>
1507
+ <td>267733.25</td>
1508
+ <td>254506.71875</td>
1509
+ </tr>
1510
+ <tr>
1511
+ <td>Metharme-7b-4bit-ao-damp0.1</td>
1512
+ <td>6.220707893371582</td>
1513
+ <td>47.82929611206055</td>
1514
+ <td>7.884565353393555</td>
1515
+ </tr>
1516
+ <tr>
1517
+ <td>Metharme-7b-4bit-ao-sym</td>
1518
+ <td>6.477842807769775</td>
1519
+ <td>48.53507614135742</td>
1520
+ <td>7.993765354156494</td>
1521
+ </tr>
1522
+ <tr>
1523
+ <td>Metharme-7b-4bit-ao</td>
1524
+ <td>6.281311511993408</td>
1525
+ <td>46.79158401489258</td>
1526
+ <td>7.906069755554199</td>
1527
+ </tr>
1528
+ <tr>
1529
+ <td>Metharme-7b-4bit-damp0.1</td>
1530
+ <td>6.495014190673828</td>
1531
+ <td>49.39763641357422</td>
1532
+ <td>8.68167781829834</td>
1533
+ </tr>
1534
+ <tr>
1535
+ <td>Metharme-7b-4bit-trits</td>
1536
+ <td>7.832409858703613</td>
1537
+ <td>55.383026123046875</td>
1538
+ <td>8.806737899780273</td>
1539
+ </tr>
1540
+ <tr>
1541
+ <td>Metharme-7b-4bit-ts-128g-damp0.1</td>
1542
+ <td>6.17774772644043</td>
1543
+ <td>46.47630310058594</td>
1544
+ <td>8.20170783996582</td>
1545
+ </tr>
1546
+ <tr>
1547
+ <td>Metharme-7b-4bit-ts-128g-sym-damp0.1</td>
1548
+ <td>6.584965705871582</td>
1549
+ <td>55.20026397705078</td>
1550
+ <td>8.268644332885742</td>
1551
+ </tr>
1552
+ <tr>
1553
+ <td>Metharme-7b-4bit-ts-128g-sym-trits-damp0.1</td>
1554
+ <td>6.225503921508789</td>
1555
+ <td>53.12746047973633</td>
1556
+ <td>8.240595817565918</td>
1557
+ </tr>
1558
+ <tr>
1559
+ <td>Metharme-7b-4bit-ts-128g-trits-damp0.1</td>
1560
+ <td>6.289113521575928</td>
1561
+ <td>46.06787109375</td>
1562
+ <td>8.122251510620117</td>
1563
+ </tr>
1564
+ <tr>
1565
+ <td>Metharme-7b-4bit-ts-128g</td>
1566
+ <td>9.475017547607422</td>
1567
+ <td>52.358829498291016</td>
1568
+ <td>8.340700149536133</td>
1569
+ </tr>
1570
+ <tr>
1571
+ <td>Metharme-7b-4bit-ts-32g-damp0.1</td>
1572
+ <td>6.219024658203125</td>
1573
+ <td>45.834869384765625</td>
1574
+ <td>8.071272850036621</td>
1575
+ </tr>
1576
+ <tr>
1577
+ <td>Metharme-7b-4bit-ts-32g-sym-damp0.1</td>
1578
+ <td>6.274552822113037</td>
1579
+ <td>47.35737228393555</td>
1580
+ <td>8.06270980834961</td>
1581
+ </tr>
1582
+ <tr>
1583
+ <td>Metharme-7b-4bit-ts-32g-sym-trits-damp0.1</td>
1584
+ <td>6.266031265258789</td>
1585
+ <td>47.346702575683594</td>
1586
+ <td>8.068148612976074</td>
1587
+ </tr>
1588
+ <tr>
1589
+ <td>Metharme-7b-4bit-ts-32g-trits-damp0.1</td>
1590
+ <td>6.118431568145752</td>
1591
+ <td>45.67333221435547</td>
1592
+ <td>8.077078819274902</td>
1593
+ </tr>
1594
+ <tr>
1595
+ <td>Metharme-7b-4bit-ts-32g</td>
1596
+ <td>6.424218654632568</td>
1597
+ <td>48.48588943481445</td>
1598
+ <td>8.089512825012207</td>
1599
+ </tr>
1600
+ <tr>
1601
+ <td>Metharme-7b-4bit-ts-damp0.1</td>
1602
+ <td>6.7783050537109375</td>
1603
+ <td>51.81301498413086</td>
1604
+ <td>8.621373176574707</td>
1605
+ </tr>
1606
+ <tr>
1607
+ <td>Metharme-7b-4bit-ts-sym-damp0.1</td>
1608
+ <td>6.7517595291137695</td>
1609
+ <td>54.06147384643555</td>
1610
+ <td>8.821818351745605</td>
1611
+ </tr>
1612
+ <tr>
1613
+ <td>Metharme-7b-4bit-ts-sym-trits-damp0.1</td>
1614
+ <td>6.886506080627441</td>
1615
+ <td>64.72743225097656</td>
1616
+ <td>8.880627632141113</td>
1617
+ </tr>
1618
+ <tr>
1619
+ <td>Metharme-7b-4bit-ts-trits-damp0.1</td>
1620
+ <td>6.631694793701172</td>
1621
+ <td>51.7371711730957</td>
1622
+ <td>8.656966209411621</td>
1623
+ </tr>
1624
+ <tr>
1625
+ <td>Metharme-7b-4bit-ts</td>
1626
+ <td>7.735969543457031</td>
1627
+ <td>62.92238235473633</td>
1628
+ <td>8.913650512695312</td>
1629
+ </tr>
1630
+ <tr>
1631
+ <td>Metharme-7b-4bit-128g-sym-trits</td>
1632
+ <td>7.056288242340088</td>
1633
+ <td>48.983768463134766</td>
1634
+ <td>8.339276313781738</td>
1635
+ </tr>
1636
+ <tr>
1637
+ <td>Metharme-7b-4bit-128g-sym</td>
1638
+ <td>6.9575653076171875</td>
1639
+ <td>49.356834411621094</td>
1640
+ <td>8.35644817352295</td>
1641
+ </tr>
1642
+ <tr>
1643
+ <td>Metharme-7b-4bit-128g-trits</td>
1644
+ <td>8.68796443939209</td>
1645
+ <td>49.73833465576172</td>
1646
+ <td>8.2406587600708</td>
1647
+ </tr>
1648
+ <tr>
1649
+ <td>Metharme-7b-4bit-32g-sym-trits</td>
1650
+ <td>6.206963539123535</td>
1651
+ <td>46.88833999633789</td>
1652
+ <td>8.13610553741455</td>
1653
+ </tr>
1654
+ <tr>
1655
+ <td>Metharme-7b-4bit-32g-sym</td>
1656
+ <td>6.518134117126465</td>
1657
+ <td>49.66925811767578</td>
1658
+ <td>8.13516616821289</td>
1659
+ </tr>
1660
+ <tr>
1661
+ <td>Metharme-7b-4bit-32g-trits</td>
1662
+ <td>7.017086029052734</td>
1663
+ <td>45.04129409790039</td>
1664
+ <td>8.074845314025879</td>
1665
+ </tr>
1666
+ <tr>
1667
+ <td>Metharme-7b-4bit-ts-128g-sym-trits</td>
1668
+ <td>6.756448745727539</td>
1669
+ <td>51.510311126708984</td>
1670
+ <td>8.292160987854004</td>
1671
+ </tr>
1672
+ <tr>
1673
+ <td>Metharme-7b-4bit-ts-128g-sym</td>
1674
+ <td>6.819341659545898</td>
1675
+ <td>55.28740310668945</td>
1676
+ <td>8.377721786499023</td>
1677
+ </tr>
1678
+ <tr>
1679
+ <td>Metharme-7b-4bit-ts-128g-trits</td>
1680
+ <td>7.350858688354492</td>
1681
+ <td>44.25314712524414</td>
1682
+ <td>8.274221420288086</td>
1683
+ </tr>
1684
+ <tr>
1685
+ <td>Metharme-7b-4bit-ts-32g-sym-trits</td>
1686
+ <td>6.82415771484375</td>
1687
+ <td>48.82029724121094</td>
1688
+ <td>8.090987205505371</td>
1689
+ </tr>
1690
+ <tr>
1691
+ <td>Metharme-7b-4bit-ts-32g-sym</td>
1692
+ <td>6.387663841247559</td>
1693
+ <td>48.07244110107422</td>
1694
+ <td>8.173730850219727</td>
1695
+ </tr>
1696
+ <tr>
1697
+ <td>Metharme-7b-4bit-ts-32g-trits</td>
1698
+ <td>6.632943153381348</td>
1699
+ <td>47.973228454589844</td>
1700
+ <td>8.013848304748535</td>
1701
+ </tr>
1702
+ <tr>
1703
+ <td>Metharme-7b-4bit-ts-sym-trits</td>
1704
+ <td>7.599876403808594</td>
1705
+ <td>55.75454330444336</td>
1706
+ <td>8.932201385498047</td>
1707
+ </tr>
1708
+ <tr>
1709
+ <td>Metharme-7b-4bit-ts-sym</td>
1710
+ <td>7.270263671875</td>
1711
+ <td>54.35262680053711</td>
1712
+ <td>8.787986755371094</td>
1713
+ </tr>
1714
+ <tr>
1715
+ <td>Metharme-7b-4bit-ts-trits</td>
1716
+ <td>7.2306809425354</td>
1717
+ <td>66.78710174560547</td>
1718
+ <td>8.879831314086914</td>
1719
+ </tr>
1720
+ <tr>
1721
+ <td>Metharme-ao-sym-trits-damp0.1</td>
1722
+ <td>6.329384803771973</td>
1723
+ <td>48.06882858276367</td>
1724
+ <td>7.959168910980225</td>
1725
+ </tr>
1726
+ <tr>
1727
+ <td>Metharme-ao-sym-trits</td>
1728
+ <td>6.390106678009033</td>
1729
+ <td>48.15375900268555</td>
1730
+ <td>7.9804582595825195</td>
1731
+ </tr>
1732
+ <tr>
1733
+ <td>Metharme-ao-sym</td>
1734
+ <td>6.477842807769775</td>
1735
+ <td>48.53507614135742</td>
1736
+ <td>7.993765354156494</td>
1737
+ </tr>
1738
+ <tr>
1739
+ <td>Metharme-ao-trits-damp0.1</td>
1740
+ <td>6.163661956787109</td>
1741
+ <td>46.50249099731445</td>
1742
+ <td>7.877425193786621</td>
1743
+ </tr>
1744
+ <tr>
1745
+ <td>Metharme-ao-trits</td>
1746
+ <td>6.283935546875</td>
1747
+ <td>46.57590103149414</td>
1748
+ <td>7.907411575317383</td>
1749
+ </tr>
1750
+ <tr>
1751
+ <td>Metharme-ao-ts-damp0.1</td>
1752
+ <td>6.184001445770264</td>
1753
+ <td>46.17180633544922</td>
1754
+ <td>7.880400657653809</td>
1755
+ </tr>
1756
+ <tr>
1757
+ <td>Metharme-ao-ts-sym-damp0.1</td>
1758
+ <td>6.210323333740234</td>
1759
+ <td>47.66789245605469</td>
1760
+ <td>7.952476978302002</td>
1761
+ </tr>
1762
+ <tr>
1763
+ <td>Metharme-ao-ts-sym-trits-damp0.1</td>
1764
+ <td>6.232576370239258</td>
1765
+ <td>48.081459045410156</td>
1766
+ <td>7.95023250579834</td>
1767
+ </tr>
1768
+ <tr>
1769
+ <td>Metharme-ao-ts-sym-trits</td>
1770
+ <td>6.471063137054443</td>
1771
+ <td>49.650611877441406</td>
1772
+ <td>7.969552040100098</td>
1773
+ </tr>
1774
+ <tr>
1775
+ <td>Metharme-ao-ts-sym</td>
1776
+ <td>6.460526943206787</td>
1777
+ <td>47.190460205078125</td>
1778
+ <td>7.9732160568237305</td>
1779
+ </tr>
1780
+ <tr>
1781
+ <td>Metharme-ao-ts-trits-damp0.1</td>
1782
+ <td>6.172733783721924</td>
1783
+ <td>45.19890213012695</td>
1784
+ <td>7.872506141662598</td>
1785
+ </tr>
1786
+ <tr>
1787
+ <td>Metharme-ao-ts-trits</td>
1788
+ <td>6.310682773590088</td>
1789
+ <td>46.4483757019043</td>
1790
+ <td>7.898126602172852</td>
1791
+ </tr>
1792
+ <tr>
1793
+ <td>Metharme-ao-ts</td>
1794
+ <td>6.329496383666992</td>
1795
+ <td>46.88129806518555</td>
1796
+ <td>7.910323143005371</td>
1797
+ </tr>
1798
+ <tr>
1799
+ <td>Metharme-sym-damp0.1</td>
1800
+ <td>6.896804332733154</td>
1801
+ <td>57.4250602722168</td>
1802
+ <td>8.703770637512207</td>
1803
+ </tr>
1804
+ <tr>
1805
+ <td>Metharme-sym-trits-damp0.1</td>
1806
+ <td>7.075908184051514</td>
1807
+ <td>59.13897705078125</td>
1808
+ <td>8.919178009033203</td>
1809
+ </tr>
1810
+ <tr>
1811
+ <td>Metharme-sym-trits</td>
1812
+ <td>7.494253635406494</td>
1813
+ <td>63.320709228515625</td>
1814
+ <td>8.969240188598633</td>
1815
+ </tr>
1816
+ <tr>
1817
+ <td>Metharme-sym</td>
1818
+ <td>7.585672855377197</td>
1819
+ <td>61.01168441772461</td>
1820
+ <td>9.032520294189453</td>
1821
+ </tr>
1822
+ <tr>
1823
+ <td>Metharme-trits</td>
1824
+ <td>7.832409858703613</td>
1825
+ <td>55.383026123046875</td>
1826
+ <td>8.806737899780273</td>
1827
+ </tr>
1828
+ </tbody>
1829
+ </table>
1830
+ </details>
1831
  </body>
1832
  </html>