Update README.md
Browse files
README.md
CHANGED
@@ -51,6 +51,15 @@ Here I explore whether training on long sequences that have clear conceptual dep
|
|
51 |
| TheBloke/airoboros-13B-gpt4-1-4-SuperHOT-8K-GPTQ | 4096 | 5.80 |
|
52 |
| **bhenrym14/airoboros-13b-gpt4-1.4.1-PI-8192-GPTQ** | 4096 | **5.15** |
|
53 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
54 |
|
55 |
|
56 |
## Quantization:
|
|
|
51 |
| TheBloke/airoboros-13B-gpt4-1-4-SuperHOT-8K-GPTQ | 4096 | 5.80 |
|
52 |
| **bhenrym14/airoboros-13b-gpt4-1.4.1-PI-8192-GPTQ** | 4096 | **5.15** |
|
53 |
|
54 |
+
| Context (tokens) | airophin-13b-pntk-16k-fp16| bhenrym14/airoboros-13b-gpt4-1.4.1-PI-8192-GPTQ |bhenrym14/airoboros-33b-gpt4-1.4.1-lxctx-PI-16384-fp16 | TheBloke/airoboros-33B-gpt4-1-4-SuperHOT-8K-GPTQ | jondurbin/airoboros-33B-gpt4-1.4-GPTQ |
|
55 |
+
| ---| ------- | -----| ------ | --- | --- |
|
56 |
+
| 512 | 7.62 | | 7.90 | 8.24 | **6.36** |
|
57 |
+
| 1024 | 6.20 | | 6.17 | 8.06 | **5.12** |
|
58 |
+
| 2048 | 5.38 | | 5.23 | 7.02 | **4.43** |
|
59 |
+
| 4096 | 5.08 | | **4.91** | 6.56 | 54.5 |
|
60 |
+
| 8192 | 4.90 | | -- | -- | -- |
|
61 |
+
| 12000 | 4.82 | | -- | -- | -- |
|
62 |
+
|
63 |
|
64 |
|
65 |
## Quantization:
|