Update README.md
Browse files
README.md
CHANGED
@@ -14,7 +14,7 @@ have fun =)
|
|
14 |
[EDIT 2] - Usage Notes - model is sorta picky with the batch size and prompt preset/template. (maybe because merge of ChatML and OpenChat models)
|
15 |
|
16 |
My current recommended setting & findings
|
17 |
-
- Using LM Studio - use the default preset. GPU acceleration to max. prompt eval size to 1024, context length to 32768. this yields me
|
18 |
- Using Oobabooga (Windows PC) - runs well using run-in-4bit along with use_flash_attention_2. default presets and everything works just fine.
|
19 |
- Using OobaBooga (Mac) - [investigating]
|
20 |
|
|
|
14 |
[EDIT 2] - Usage Notes - model is sorta picky with the batch size and prompt preset/template. (maybe because merge of ChatML and OpenChat models)
|
15 |
|
16 |
My current recommended setting & findings
|
17 |
+
- Using LM Studio - use the default preset. GPU acceleration to max. prompt eval size to 1024, context length to 32768. this yields me decent, coherant results. ChatML works too but occasionall spits up odd texts after a couple of turns.
|
18 |
- Using Oobabooga (Windows PC) - runs well using run-in-4bit along with use_flash_attention_2. default presets and everything works just fine.
|
19 |
- Using OobaBooga (Mac) - [investigating]
|
20 |
|