Update README.md
Browse files
README.md
CHANGED
@@ -13,6 +13,16 @@ The source code can also be used directly.
|
|
13 |
|
14 |
NOTE: Links to GGUFs below.
|
15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
<B>IMPORTANT: Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers</B>
|
17 |
|
18 |
If you are going to use this model, (source, GGUF or a different quant), please review this document for critical parameter, sampler and advance sampler settings (for multiple AI/LLM aps).
|
|
|
13 |
|
14 |
NOTE: Links to GGUFs below.
|
15 |
|
16 |
+
MOE SPECIFIC NOTE:
|
17 |
+
|
18 |
+
If you want to change the "default" number of experts set, modify the "config.json" :
|
19 |
+
|
20 |
+
"num_experts_per_tok": 2,
|
21 |
+
|
22 |
+
The user will still be able to modify it, if the LLM/AI app has the setting option to do this.
|
23 |
+
|
24 |
+
Each time you add/subtract an expert the token per second speed will change.
|
25 |
+
|
26 |
<B>IMPORTANT: Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers</B>
|
27 |
|
28 |
If you are going to use this model, (source, GGUF or a different quant), please review this document for critical parameter, sampler and advance sampler settings (for multiple AI/LLM aps).
|