llmixer commited on
Commit
b1b1b9f
1 Parent(s): 7d3e8cd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +47 -0
README.md CHANGED
@@ -1,3 +1,50 @@
1
  ---
2
  license: llama2
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: llama2
3
+ language:
4
+ - en
5
+ pipeline_tag: conversational
6
+ tags:
7
+ - aurelian
8
+ - WinterGoddess
9
+ - frankenmerge
10
+ - 120b
11
  ---
12
+ # BigAurelian v0.5 120b 32k
13
+ A Goliath-120b style frankenmerge of aurelian-v0.5-70b-32K and WinterGoddess-1.4x-70b. Meant as a slight update to Goliath by using the successor model to Euryale.
14
+
15
+ # Prompting Format
16
+ Llama2 and Alpaca.
17
+
18
+ # Merge process
19
+ The models used in the merge are [aurelian-v0.5-70b-32K](https://huggingface.co/grimulkan/aurelian-v0.5-70b-rope8-32K-fp16) and [WinterGoddess-1.4x-70b](https://huggingface.co/Sao10K/WinterGoddess-1.4x-70B-L2).
20
+
21
+ The layer mix:
22
+ ```yaml
23
+ - range 0, 16
24
+ aurelian
25
+ - range 8, 24
26
+ WinterGoddess
27
+ - range 17, 32
28
+ aurelian
29
+ - range 25, 40
30
+ WinterGoddess
31
+ - range 33, 48
32
+ aurelian
33
+ - range 41, 56
34
+ WinterGoddess
35
+ - range 49, 64
36
+ aurelian
37
+ - range 57, 72
38
+ WinterGoddess
39
+ - range 65, 80
40
+ aurelian
41
+ ```
42
+
43
+ # Acknowledgements
44
+ [@grimulkan](https://huggingface.co/grimulkan) For creating grimulkan
45
+
46
+ [@Sao10K](https://huggingface.co/Sao10K) For creating WinterGoddess
47
+
48
+ [@alpindale](https://huggingface.co/alpindale) For creating the original Goliath
49
+
50
+ [@chargoddard](https://huggingface.co/chargoddard) For developing [mergekit](https://github.com/cg123/mergekit).