femiari commited on
Commit
e1bdcd6
1 Parent(s): f38e7bf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -25
README.md CHANGED
@@ -21,31 +21,6 @@ QwenMoEAriel is a Mixture of Experts (MoE) made with the following models using
21
 
22
  ## 🧩 Configuration
23
 
24
- ```yaml
25
- base_model: Qwen/Qwen2-1.5B
26
- architecture: qwen
27
- experts:
28
- - source_model: Qwen/Qwen2-1.5B
29
- positive_prompts:
30
- - "chat"
31
- - "assistant"
32
- - "tell me"
33
- - "explain"
34
- - "I want"
35
- - source_model: Replete-AI/Replete-Coder-Qwen2-1.5b
36
- positive_prompts:
37
- - "code"
38
- - "python"
39
- - "javascript"
40
- - "programming"
41
- - "algorithm"
42
- shared_experts:
43
- - source_model: Qwen/Qwen2-1.5B
44
- positive_prompts: # required by Qwen MoE for "hidden" gate mode, otherwise not allowed
45
- - "chat"
46
- # (optional, but recommended:)
47
- residual_scale: 0.1 # downweight output from shared expert to prevent overcooking the model
48
- ```
49
 
50
  ## 💻 Usage
51
 
 
21
 
22
  ## 🧩 Configuration
23
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
  ## 💻 Usage
26