Update README.md
Browse files
README.md
CHANGED
@@ -17,7 +17,7 @@ Dataset Modifications:
|
|
17 |
<br>\- More Creative Writing Samples -> 2x
|
18 |
<br>\- Remade and Refined Detailed Instruct Data
|
19 |
|
20 |
-
Coherent at 32K Context. Obviously not as good as a native 32K Context Model, but good enough. Has some of the usual memory
|
21 |
|
22 |
Sanity Check // Needle in a Haystack Results:
|
23 |
![Results](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.3-32K/resolve/main/haystack.png)
|
|
|
17 |
<br>\- More Creative Writing Samples -> 2x
|
18 |
<br>\- Remade and Refined Detailed Instruct Data
|
19 |
|
20 |
+
Coherent at 32K Context. Obviously not as good as a native 32K Context Model, but good enough. Has some of the usual memory issues in the middle of context, it has some problems with long-context understanding and reasoning, but it does not break down into incoherency like regular rope scaling does.
|
21 |
|
22 |
Sanity Check // Needle in a Haystack Results:
|
23 |
![Results](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.3-32K/resolve/main/haystack.png)
|