theBodhiTree commited on
Commit
f50e7db
1 Parent(s): 83fef88

updated readme

Browse files
Files changed (1) hide show
  1. README.md +45 -3
README.md CHANGED
@@ -1,3 +1,45 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - HuggingFaceH4/zephyr-7b-alpha
4
+ - HuggingFaceH4/zephyr-7b-beta
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+
12
+ ## Merge Details
13
+ ### Merge Method
14
+
15
+ This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [HuggingFaceH4/zephyr-7b-alpha](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha) as a base.
16
+
17
+ ### Models Merged
18
+
19
+ The following models were included in the merge:
20
+ * [HuggingFaceH4/zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
21
+ * [HuggingFaceH4/zephyr-7b-alpha](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha)
22
+
23
+ ### Configuration
24
+
25
+ The following YAML configuration was used to produce this model:
26
+
27
+ ```yaml
28
+
29
+ dtype: bfloat16
30
+ merge_method: task_arithmetic
31
+ base_model:
32
+ model: HuggingFaceH4/zephyr-7b-alpha
33
+ slices:
34
+ - sources:
35
+ - layer_range: [0, 32]
36
+ model: HuggingFaceH4/zephyr-7b-beta
37
+ parameters:
38
+ weight: 0.5
39
+ - layer_range: [0, 32]
40
+ model: HuggingFaceH4/zephyr-7b-alpha
41
+ parameters:
42
+ weight: 0.5
43
+
44
+
45
+ ```