results/results.txt

|    |   accuracy |   bits |   clip | dataset   | method   | model   | params      |   peak_mem_alloc_mb |   peak_mem_reserv_mb |   perplexity |   time_ms |   wer |
|---:|-----------:|-------:|-------:|:----------|:---------|:--------|:------------|--------------------:|---------------------:|-------------:|----------:|------:|
|  0 |     0.4278 |      4 |      0 | wikitext  | ao_woq   | llama3  | fixed       |             5542.36 |              7163.87 |       5.3584 |   39.1541 |     0 |
|  1 |     0      |      4 |      0 | wikitext  | comb_3   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.2404 |  121.29   |     0 |
|  2 |     0      |      4 |      0 | wikitext  | comb_7   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.1248 |  145.772  |     0 |
|  3 |     0.4107 |      4 |      0 | wikitext  | comb_15  | llama3  | rnd_naive_t |            14179.2  |             15367.9  |       6.0225 |   68.5664 |     0 |
|  4 |     0.3798 |      4 |      0 | wikitext  | woq      | llama3  | fixed       |            14179.3  |             15367.9  |       6.1632 |   95.4438 |     0 |
|  5 |     0      |      4 |      0 | wikitext  | comb_5   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.1396 |  147.454  |     0 |
|  6 |     0      |      4 |      0 | wikitext  | comb_4   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.5297 |  147.614  |     0 |
|  7 |     0      |      4 |      0 | wikitext  | comb_16  | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.5022 |  148.156  |     0 |
|  8 |     0      |      4 |      0 | wikitext  | comb_2   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |      21.7492 |  201.147  |     0 |
|  9 |     0      |      4 |      0 | wikitext  | comb_8   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.4968 |  201.532  |     0 |
| 10 |     0.3953 |      4 |      0 | wikitext  | comb_18  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       6.4111 |  147.345  |     0 |
| 11 |     0.3991 |      4 |      0 | wikitext  | comb_17  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       6.3314 |  147.383  |     0 |
| 12 |     0.4024 |      4 |      0 | wikitext  | comb_11  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       6.0226 |  147.729  |     0 |
| 13 |     0.3888 |      4 |      0 | wikitext  | comb_1   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.833  |  147.793  |     0 |
| 14 |     0.396  |      4 |      0 | wikitext  | comb_6   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       6.2106 |  175.215  |     0 |
| 15 |     0.3873 |      4 |      0 | wikitext  | comb_9   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       6.1515 |  200.883  |     0 |
| 16 |     0      |      4 |      0 | wikitext  | comb_14  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1356 |    4.3997 |     0 |
| 17 |     0      |      4 |      0 | wikitext  | comb_10  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1248 |    4.4001 |     0 |
| 18 |     0      |      4 |      0 | wikitext  | comb_13  | llama3  | rnd_naive_t |            16812    |             19811.8  |       1.1442 |  426.279  |     0 |
| 19 |     0      |      4 |      0 | wikitext  | comb_12  | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.1387 |    4.014  |     0 |
| 20 |     0      |      4 |      0 | wikitext  | comb_0   | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.1816 |    4.026  |     0 |
|  0 |     0      |      7 |      0 | wikitext  | comb_3   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.1227 |  121.386  |     0 |
|  1 |     0      |      7 |      0 | wikitext  | comb_7   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.1248 |  146.378  |     0 |
|  2 |     0.4433 |      7 |      0 | wikitext  | comb_15  | llama3  | rnd_naive_t |            14179.2  |             15367.9  |       5.2359 |   68.7039 |     0 |
|  3 |     0.4428 |      7 |      0 | wikitext  | woq      | llama3  | fixed       |            14179.3  |             15367.9  |       5.2489 |   95.6709 |     0 |
|  4 |     0      |      7 |      0 | wikitext  | comb_4   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.1266 |  147.648  |     0 |
|  5 |     0      |      7 |      0 | wikitext  | comb_5   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.1346 |  147.819  |     0 |
|  6 |     0      |      7 |      0 | wikitext  | comb_16  | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.6101 |  148.15   |     0 |
|  7 |     0      |      7 |      0 | wikitext  | comb_2   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |  305327      |  201.572  |     0 |
|  8 |     0      |      7 |      0 | wikitext  | comb_8   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.343  |  201.759  |     0 |
|  9 |     0.4157 |      7 |      0 | wikitext  | comb_18  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       6.0776 |  147.72   |     0 |
| 10 |     0.4396 |      7 |      0 | wikitext  | comb_11  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.2046 |  147.977  |     0 |
| 11 |     0.4156 |      7 |      0 | wikitext  | comb_17  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.8684 |  148.034  |     0 |
| 12 |     0.4362 |      7 |      0 | wikitext  | comb_1   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.4774 |  148.296  |     0 |
| 13 |     0.4371 |      7 |      0 | wikitext  | comb_6   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.2481 |  175.323  |     0 |
| 14 |     0.4375 |      7 |      0 | wikitext  | comb_9   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.3521 |  200.984  |     0 |
| 15 |     0      |      7 |      0 | wikitext  | comb_14  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1271 |    4.4036 |     0 |
| 16 |     0      |      7 |      0 | wikitext  | comb_10  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1248 |    4.4057 |     0 |
| 17 |     0      |      7 |      0 | wikitext  | comb_13  | llama3  | rnd_naive_t |            16812    |             19811.8  |       1.1424 |  426.055  |     0 |
| 18 |     0      |      7 |      0 | wikitext  | comb_0   | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.191  |    4.0245 |     0 |
| 19 |     0      |      7 |      0 | wikitext  | comb_12  | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.1273 |    4.0303 |     0 |
|  0 |     0      |      6 |      0 | wikitext  | comb_3   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.1456 |  121.303  |     0 |
|  1 |     0      |      6 |      0 | wikitext  | comb_7   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.1248 |  146.051  |     0 |
|  2 |     0.4432 |      6 |      0 | wikitext  | comb_15  | llama3  | rnd_naive_t |            14179.2  |             15367.9  |       5.1949 |   68.7089 |     0 |
|  3 |     0.4409 |      6 |      0 | wikitext  | woq      | llama3  | fixed       |            14179.3  |             15367.9  |       5.1933 |   95.4909 |     0 |
|  4 |     0      |      6 |      0 | wikitext  | comb_4   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.2937 |  147.412  |     0 |
|  5 |     0      |      6 |      0 | wikitext  | comb_5   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.1415 |  147.57   |     0 |
|  6 |     0      |      6 |      0 | wikitext  | comb_16  | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.3004 |  148.724  |     0 |
|  7 |     0      |      6 |      0 | wikitext  | comb_2   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       2.0679 |  201.273  |     0 |
|  8 |     0      |      6 |      0 | wikitext  | comb_8   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.7716 |  201.507  |     0 |
|  9 |     0.4347 |      6 |      0 | wikitext  | comb_18  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.3582 |  147.317  |     0 |
| 10 |     0.4289 |      6 |      0 | wikitext  | comb_11  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.3364 |  147.431  |     0 |
| 11 |     0.4362 |      6 |      0 | wikitext  | comb_17  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.2435 |  147.669  |     0 |
| 12 |     0.4361 |      6 |      0 | wikitext  | comb_1   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.193  |  148.371  |     0 |
| 13 |     0.4388 |      6 |      0 | wikitext  | comb_6   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.195  |  175.111  |     0 |
| 14 |     0.4375 |      6 |      0 | wikitext  | comb_9   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.2482 |  200.905  |     0 |
| 15 |     0      |      6 |      0 | wikitext  | comb_14  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1354 |    4.3939 |     0 |
| 16 |     0      |      6 |      0 | wikitext  | comb_10  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1248 |    4.3968 |     0 |
| 17 |     0      |      6 |      0 | wikitext  | comb_13  | llama3  | rnd_naive_t |            16812    |             19811.8  |       1.1368 |  426.215  |     0 |
| 18 |     0      |      6 |      0 | wikitext  | comb_0   | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.2187 |    4.0246 |     0 |
| 19 |     0      |      6 |      0 | wikitext  | comb_12  | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.1379 |    4.0247 |     0 |
|  0 |     0.4437 |      8 |      0 | wikitext  | ao_woq   | llama3  | fixed       |            12110.4  |             14040.4  |       5.2086 |   52.4367 |     0 |
|  1 |     0      |      8 |      0 | wikitext  | comb_3   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.1235 |  121.23   |     0 |
|  2 |     0      |      8 |      0 | wikitext  | comb_7   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.1248 |  145.654  |     0 |
|  3 |     0.4431 |      8 |      0 | wikitext  | comb_15  | llama3  | rnd_naive_t |            14179.2  |             15367.9  |       5.2079 |   68.5261 |     0 |
|  4 |     0.4439 |      8 |      0 | wikitext  | woq      | llama3  | fixed       |            14179.3  |             15367.9  |       5.2082 |   95.5066 |     0 |
|  5 |     0      |      8 |      0 | wikitext  | comb_4   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.1249 |  147.482  |     0 |
|  6 |     0      |      8 |      0 | wikitext  | comb_5   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.149  |  147.537  |     0 |
|  7 |     0      |      8 |      0 | wikitext  | comb_16  | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       2.0685 |  147.813  |     0 |
|  8 |     0      |      8 |      0 | wikitext  | comb_8   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       2.071  |  201.287  |     0 |
|  9 |     0      |      8 |      0 | wikitext  | comb_2   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       2.8023 |  201.344  |     0 |
| 10 |     0.4377 |      8 |      0 | wikitext  | comb_18  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.2764 |  147.449  |     0 |
| 11 |     0.4405 |      8 |      0 | wikitext  | comb_17  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.3516 |  147.472  |     0 |
| 12 |     0.4364 |      8 |      0 | wikitext  | comb_11  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.4312 |  147.864  |     0 |
| 13 |     0.4338 |      8 |      0 | wikitext  | comb_1   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.4813 |  147.988  |     0 |
| 14 |     0.4386 |      8 |      0 | wikitext  | comb_6   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.4114 |  174.892  |     0 |
| 15 |     0.4383 |      8 |      0 | wikitext  | comb_9   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.4026 |  201.034  |     0 |
| 16 |     0      |      8 |      0 | wikitext  | comb_10  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1248 |    4.3883 |     0 |
| 17 |     0      |      8 |      0 | wikitext  | comb_14  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1406 |    4.3931 |     0 |
| 18 |     0      |      8 |      0 | wikitext  | comb_13  | llama3  | rnd_naive_t |            16812    |             19811.8  |       1.2057 |  426.279  |     0 |
| 19 |     0      |      8 |      0 | wikitext  | comb_12  | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.1393 |    4.0152 |     0 |
| 20 |     0      |      8 |      0 | wikitext  | comb_0   | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.2073 |    4.0267 |     0 |
|  0 |     0      |      5 |      0 | wikitext  | comb_3   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.22   |  121.553  |     0 |
|  1 |     0      |      5 |      0 | wikitext  | comb_7   | llama3  | rnd_naive_t |            14178.7  |             15367.9  |       1.1248 |  146.012  |     0 |
|  2 |     0.4349 |      5 |      0 | wikitext  | comb_15  | llama3  | rnd_naive_t |            14179.2  |             15367.9  |       5.2777 |   68.6646 |     0 |
|  3 |     0.4356 |      5 |      0 | wikitext  | woq      | llama3  | fixed       |            14179.3  |             15367.9  |       5.449  |   95.562  |     0 |
|  4 |     0      |      5 |      0 | wikitext  | comb_4   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.1254 |  147.496  |     0 |
|  5 |     0      |      5 |      0 | wikitext  | comb_5   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.1356 |  147.619  |     0 |
|  6 |     0      |      5 |      0 | wikitext  | comb_16  | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.9329 |  147.785  |     0 |
|  7 |     0      |      5 |      0 | wikitext  | comb_2   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |     202.672  |  201.198  |     0 |
|  8 |     0      |      5 |      0 | wikitext  | comb_8   | llama3  | rnd_naive_t |            14185.3  |             15374.2  |       1.1547 |  201.25   |     0 |
|  9 |     0.4259 |      5 |      0 | wikitext  | comb_18  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.4124 |  147.422  |     0 |
| 10 |     0.4286 |      5 |      0 | wikitext  | comb_17  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.1816 |  147.843  |     0 |
| 11 |     0.4327 |      5 |      0 | wikitext  | comb_11  | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.3567 |  147.881  |     0 |
| 12 |     0.4308 |      5 |      0 | wikitext  | comb_1   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.2541 |  148.199  |     0 |
| 13 |     0.4311 |      5 |      0 | wikitext  | comb_6   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.2835 |  175.096  |     0 |
| 14 |     0.4292 |      5 |      0 | wikitext  | comb_9   | llama3  | rnd_naive_t |            14185.7  |             15374.2  |       5.3046 |  200.922  |     0 |
| 15 |     0      |      5 |      0 | wikitext  | comb_14  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1407 |    4.3913 |     0 |
| 16 |     0      |      5 |      0 | wikitext  | comb_10  | llama3  | rnd_naive_t |            11051.9  |             18748.5  |       1.1248 |    4.4017 |     0 |
| 17 |     0      |      5 |      0 | wikitext  | comb_13  | llama3  | rnd_naive_t |            16812    |             19811.8  |       1.1501 |  425.965  |     0 |
| 18 |     0      |      5 |      0 | wikitext  | comb_0   | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.2129 |    3.9972 |     0 |
| 19 |     0      |      5 |      0 | wikitext  | comb_12  | llama3  | rnd_naive_t |            11051.9  |             23421    |       1.1248 |    4.0307 |     0 |
| 21 |     0.4437 |     32 |      0 | wikitext  | base     | llama3  | ----        |            33003.5  |             33156    |       5.2238 |   26.1508 |     0 |