Sao10K commited on
Commit
c2c0399
·
verified ·
1 Parent(s): 128ed6e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -17,7 +17,7 @@ Dataset Modifications:
17
  <br>\- More Creative Writing Samples -> 2x
18
  <br>\- Remade and Refined Detailed Instruct Data
19
 
20
- Coherent at 32K Context. Obviously not as good as a native 32K Context Model, but good enough. Has some of the usual memory recall issues in the middle of context, but it does not break down into incoherency like regular rope scaling does.
21
 
22
  Sanity Check // Needle in a Haystack Results:
23
  ![Results](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.3-32K/resolve/main/haystack.png)
 
17
  <br>\- More Creative Writing Samples -> 2x
18
  <br>\- Remade and Refined Detailed Instruct Data
19
 
20
+ Coherent at 32K Context. Obviously not as good as a native 32K Context Model, but good enough. Has some of the usual memory issues in the middle of context, it has some problems with long-context understanding and reasoning, but it does not break down into incoherency like regular rope scaling does.
21
 
22
  Sanity Check // Needle in a Haystack Results:
23
  ![Results](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.3-32K/resolve/main/haystack.png)