abideen commited on
Commit
2dd6f3f
1 Parent(s): 0419515

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -48
README.md CHANGED
@@ -2,63 +2,16 @@
2
  tags:
3
  - merge
4
  - mergekit
5
- - lazymergekit
6
- - Qwen/Qwen1.5-72B
7
- - Qwen/Qwen1.5-72B
8
- - Qwen/Qwen1.5-72B
9
- - Qwen/Qwen1.5-72B
10
- - Qwen/Qwen1.5-72B
11
- - Qwen/Qwen1.5-72B
12
  - Qwen/Qwen1.5-72B
13
  base_model:
14
  - Qwen/Qwen1.5-72B
15
- - Qwen/Qwen1.5-72B
16
- - Qwen/Qwen1.5-72B
17
- - Qwen/Qwen1.5-72B
18
- - Qwen/Qwen1.5-72B
19
- - Qwen/Qwen1.5-72B
20
- - Qwen/Qwen1.5-72B
21
  ---
22
 
23
  # Qwen-120B
24
 
25
- Qwen-120B is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
26
- * [Qwen/Qwen1.5-72B](https://huggingface.co/Qwen/Qwen1.5-72B)
27
- * [Qwen/Qwen1.5-72B](https://huggingface.co/Qwen/Qwen1.5-72B)
28
- * [Qwen/Qwen1.5-72B](https://huggingface.co/Qwen/Qwen1.5-72B)
29
  * [Qwen/Qwen1.5-72B](https://huggingface.co/Qwen/Qwen1.5-72B)
30
- * [Qwen/Qwen1.5-72B](https://huggingface.co/Qwen/Qwen1.5-72B)
31
- * [Qwen/Qwen1.5-72B](https://huggingface.co/Qwen/Qwen1.5-72B)
32
- * [Qwen/Qwen1.5-72B](https://huggingface.co/Qwen/Qwen1.5-72B)
33
-
34
- ## 🧩 Configuration
35
 
36
- ```yaml
37
- dtype: float16
38
- merge_method: passthrough
39
- slices:
40
- - sources:
41
- - layer_range: [0, 20]
42
- model: Qwen/Qwen1.5-72B
43
- - sources:
44
- - layer_range: [10, 30]
45
- model: Qwen/Qwen1.5-72B
46
- - sources:
47
- - layer_range: [20, 40]
48
- model: Qwen/Qwen1.5-72B
49
- - sources:
50
- - layer_range: [30, 50]
51
- model: Qwen/Qwen1.5-72B
52
- - sources:
53
- - layer_range: [40, 60]
54
- model: Qwen/Qwen1.5-72B
55
- - sources:
56
- - layer_range: [50, 70]
57
- model: Qwen/Qwen1.5-72B
58
- - sources:
59
- - layer_range: [60, 80]
60
- model: Qwen/Qwen1.5-72B
61
- ```
62
 
63
  ## 💻 Usage
64
 
 
2
  tags:
3
  - merge
4
  - mergekit
 
 
 
 
 
 
 
5
  - Qwen/Qwen1.5-72B
6
  base_model:
7
  - Qwen/Qwen1.5-72B
 
 
 
 
 
 
8
  ---
9
 
10
  # Qwen-120B
11
 
12
+ Qwen-120B is a passthrough merge of the following models:
 
 
 
13
  * [Qwen/Qwen1.5-72B](https://huggingface.co/Qwen/Qwen1.5-72B)
 
 
 
 
 
14
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
  ## 💻 Usage
17