matchaaaaa commited on
Commit
fa4f53e
1 Parent(s): d27118d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -1
README.md CHANGED
@@ -67,12 +67,26 @@ slices:
67
  - model: SanjiWatsuki/Kunoichi-7B
68
  layer_range: [8, 16]
69
  - sources:
70
- - model: Mytho-Lemon-11B # my own merge of MythoMist-7B and LemonadeRP-4.5.3
71
  layer_range: [8, 48]
72
  merge_method: passthrough
73
  dtype: bfloat16
 
 
 
74
 
 
 
 
 
 
 
 
 
 
 
75
  ```
 
76
  It's a lot better than v1 :skull:
77
 
78
  So, the idea was to start with Fimbulvetr-11B-v2, a super solid RP model that punches wayyy above its weight especially for its coherence, reasoning, and even spatial awareness. Keeping the layers intact apparently is somewhat unusual, but I wanted to keep it closest to the input layers. I thought it would improve logic and open the door for more creativity later in the stack. I added Kunoichi next for its context and instruction following skills. This worked very well in v1. Lastly, I used a frankenmerge of MythoMist and LemonadeRP for the last layers. These are pretty creative models with solid writing. MythoMist in theory would give the model flavor and verbosity. LemonadeRP was recommended by a friend, and I thought it really complimented the rest of the mix quite nicely!
 
67
  - model: SanjiWatsuki/Kunoichi-7B
68
  layer_range: [8, 16]
69
  - sources:
70
+ - model: Mytho-Lemon-11B # my own merge (see below).
71
  layer_range: [8, 48]
72
  merge_method: passthrough
73
  dtype: bfloat16
74
+ ```
75
+
76
+ And here's Mytho-Lemon-11B. Yep, named it backwards.
77
 
78
+ ```yaml
79
+ slices:
80
+ - sources:
81
+ - model: KatyTheCutie/LemonadeRP-4.5.3
82
+ layer_range: [0, 24]
83
+ - sources:
84
+ - model: Gryphe/MythoMist-7B # manually added tokenizer files
85
+ layer_range: [8, 32]
86
+ merge_method: passthrough
87
+ dtype: bfloat16
88
  ```
89
+
90
  It's a lot better than v1 :skull:
91
 
92
  So, the idea was to start with Fimbulvetr-11B-v2, a super solid RP model that punches wayyy above its weight especially for its coherence, reasoning, and even spatial awareness. Keeping the layers intact apparently is somewhat unusual, but I wanted to keep it closest to the input layers. I thought it would improve logic and open the door for more creativity later in the stack. I added Kunoichi next for its context and instruction following skills. This worked very well in v1. Lastly, I used a frankenmerge of MythoMist and LemonadeRP for the last layers. These are pretty creative models with solid writing. MythoMist in theory would give the model flavor and verbosity. LemonadeRP was recommended by a friend, and I thought it really complimented the rest of the mix quite nicely!