Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,12 @@ have fun =)
|
|
| 11 |
|
| 12 |
|
| 13 |
[EDIT] - preset wise it seems like it likes the "ChatML" format.
|
|
|
|
| 14 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
---
|
| 17 |
license: apache-2.0
|
|
|
|
| 11 |
|
| 12 |
|
| 13 |
[EDIT] - preset wise it seems like it likes the "ChatML" format.
|
| 14 |
+
[EDIT 2] - Usage Notes - model is sorta picky with the batch size and prompt preset/template. (maybe because merge of ChatML and OpenChat models)
|
| 15 |
|
| 16 |
+
My current recommended setting & findings
|
| 17 |
+
- Using LM Studio - use the default preset. GPU acceleration to max. prompt eval size to 1024, context length to 32768. this yields me good, coherant results
|
| 18 |
+
- Using Oobabooga (Windows PC) - runs well using run-in-4bit along with use_flash_attention_2. default presets and everything works just fine.
|
| 19 |
+
- Using OobaBooga (Mac) - [investigating]
|
| 20 |
|
| 21 |
---
|
| 22 |
license: apache-2.0
|