matchaaaaa
commited on
Commit
•
fa4f53e
1
Parent(s):
d27118d
Update README.md
Browse files
README.md
CHANGED
@@ -67,12 +67,26 @@ slices:
|
|
67 |
- model: SanjiWatsuki/Kunoichi-7B
|
68 |
layer_range: [8, 16]
|
69 |
- sources:
|
70 |
-
- model: Mytho-Lemon-11B # my own merge
|
71 |
layer_range: [8, 48]
|
72 |
merge_method: passthrough
|
73 |
dtype: bfloat16
|
|
|
|
|
|
|
74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
75 |
```
|
|
|
76 |
It's a lot better than v1 :skull:
|
77 |
|
78 |
So, the idea was to start with Fimbulvetr-11B-v2, a super solid RP model that punches wayyy above its weight especially for its coherence, reasoning, and even spatial awareness. Keeping the layers intact apparently is somewhat unusual, but I wanted to keep it closest to the input layers. I thought it would improve logic and open the door for more creativity later in the stack. I added Kunoichi next for its context and instruction following skills. This worked very well in v1. Lastly, I used a frankenmerge of MythoMist and LemonadeRP for the last layers. These are pretty creative models with solid writing. MythoMist in theory would give the model flavor and verbosity. LemonadeRP was recommended by a friend, and I thought it really complimented the rest of the mix quite nicely!
|
|
|
67 |
- model: SanjiWatsuki/Kunoichi-7B
|
68 |
layer_range: [8, 16]
|
69 |
- sources:
|
70 |
+
- model: Mytho-Lemon-11B # my own merge (see below).
|
71 |
layer_range: [8, 48]
|
72 |
merge_method: passthrough
|
73 |
dtype: bfloat16
|
74 |
+
```
|
75 |
+
|
76 |
+
And here's Mytho-Lemon-11B. Yep, named it backwards.
|
77 |
|
78 |
+
```yaml
|
79 |
+
slices:
|
80 |
+
- sources:
|
81 |
+
- model: KatyTheCutie/LemonadeRP-4.5.3
|
82 |
+
layer_range: [0, 24]
|
83 |
+
- sources:
|
84 |
+
- model: Gryphe/MythoMist-7B # manually added tokenizer files
|
85 |
+
layer_range: [8, 32]
|
86 |
+
merge_method: passthrough
|
87 |
+
dtype: bfloat16
|
88 |
```
|
89 |
+
|
90 |
It's a lot better than v1 :skull:
|
91 |
|
92 |
So, the idea was to start with Fimbulvetr-11B-v2, a super solid RP model that punches wayyy above its weight especially for its coherence, reasoning, and even spatial awareness. Keeping the layers intact apparently is somewhat unusual, but I wanted to keep it closest to the input layers. I thought it would improve logic and open the door for more creativity later in the stack. I added Kunoichi next for its context and instruction following skills. This worked very well in v1. Lastly, I used a frankenmerge of MythoMist and LemonadeRP for the last layers. These are pretty creative models with solid writing. MythoMist in theory would give the model flavor and verbosity. LemonadeRP was recommended by a friend, and I thought it really complimented the rest of the mix quite nicely!
|