Update README.md
Browse files
README.md
CHANGED
|
@@ -48,7 +48,7 @@ Model creator: [trollek](https://huggingface.co/trollek)
|
|
| 48 |
[8bpw h8](https://huggingface.co/cgus/NinjaMouse-3B-40L-danube-exl2/tree/8bpw-h8)
|
| 49 |
|
| 50 |
## Quantization notes
|
| 51 |
-
Made with exllamav2 0.0.15 with default dataset. I'm very unsure about this model.
|
| 52 |
For me it breaks past 3000 context, at about 3500 or so. Both with these quants and the creator's GGUF files.
|
| 53 |
At first I thought I had some quantization issues but it's probably just the model itself.
|
| 54 |
|
|
|
|
| 48 |
[8bpw h8](https://huggingface.co/cgus/NinjaMouse-3B-40L-danube-exl2/tree/8bpw-h8)
|
| 49 |
|
| 50 |
## Quantization notes
|
| 51 |
+
Made with exllamav2 0.0.15 with default dataset. I'm very unsure about context size of this model.
|
| 52 |
For me it breaks past 3000 context, at about 3500 or so. Both with these quants and the creator's GGUF files.
|
| 53 |
At first I thought I had some quantization issues but it's probably just the model itself.
|
| 54 |
|