Update README.md
Browse files
README.md
CHANGED
@@ -1,16 +1,11 @@
|
|
1 |
-
---
|
2 |
-
datasets:
|
3 |
-
- PygmalionAI/PIPPA
|
4 |
-
- ludis/geepeetee4
|
5 |
-
---
|
6 |
-
|
7 |
## GGUF
|
8 |
|
9 |
gguf quants for ludis/tsukasa-13b-qlora-limarp
|
10 |
|
|
|
11 |
## Prompting
|
12 |
|
13 |
-
https://rentry.org/
|
14 |
|
15 |
The current model version has been trained on prompts using three different roles, which are denoted by the following tokens: `<|system|>`, `<|user|>` and `<|model|>`.
|
16 |
|
@@ -18,13 +13,12 @@ The `<|system|>` prompt can be used to inject out-of-channel information behind
|
|
18 |
|
19 |
## Training
|
20 |
|
21 |
-
base model (
|
22 |
-
|
23 |
-
tuned on koishi dataset (commit c83d922) for 1 epoch
|
24 |
|
25 |
-
|
|
|
26 |
|
27 |
-
|
28 |
|
29 |
-
|
30 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
## GGUF
|
2 |
|
3 |
gguf quants for ludis/tsukasa-13b-qlora-limarp
|
4 |
|
5 |
+
|
6 |
## Prompting
|
7 |
|
8 |
+
https://rentry.org/tsukasa13b - reccomended prompts and gen settings
|
9 |
|
10 |
The current model version has been trained on prompts using three different roles, which are denoted by the following tokens: `<|system|>`, `<|user|>` and `<|model|>`.
|
11 |
|
|
|
13 |
|
14 |
## Training
|
15 |
|
16 |
+
base model (mistral-0.1-7b)
|
|
|
|
|
17 |
|
18 |
+
[axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) was used for training
|
19 |
+
on a 4x nvidia a40 gpu cluster.
|
20 |
|
21 |
+
the a40 GPU cluster has been graciously provided by [Arc Compute](https://www.arccompute.io/).
|
22 |
|
23 |
+
rank 8 lora tune of mistralai/Mistral-7B-v0.1, first tuned on koishi commit 6e675d1 for one epoch then on limarp (without ponyville, lolicit, all the fallen, and eka's portal subsets) Version 2023-09-30 for 2 epochs in metharme format
|
24 |
|