ibivibiv commited on
Commit
dc19756
1 Parent(s): 59b0524

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -11,6 +11,18 @@ tags:
11
 
12
  I like to call this model "The little professor". It is simply a MOE merge of lora merged models across Llama2 and Mistral. I am using this as a test case to move to larger models and get my gate discrimination set correctly. This model is best suited for knowledge related use cases, I did not give it a specific workload target as I did with some of the other models in the "Owl Series".
13
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  # Prompting
15
 
16
  ## Prompt Template for alpaca style
 
11
 
12
  I like to call this model "The little professor". It is simply a MOE merge of lora merged models across Llama2 and Mistral. I am using this as a test case to move to larger models and get my gate discrimination set correctly. This model is best suited for knowledge related use cases, I did not give it a specific workload target as I did with some of the other models in the "Owl Series".
13
 
14
+ This model is merged from the following sources:
15
+
16
+ [Westlake-7B](https://huggingface.co/senseable/Westlake-7B)
17
+ [WestLake-7B-v2](https://huggingface.co/senseable/WestLake-7B-v2)
18
+ [openchat-nectar-0.5](https://huggingface.co/andysalerno/openchat-nectar-0.5)
19
+ [WestSeverus-7B-DPO-v2](https://huggingface.co/FelixChao/WestSeverus-7B-DPO-v2)
20
+ [WestSeverus-7B-DPO](https://huggingface.co/PetroGPT/WestSeverus-7B-DPO)
21
+
22
+ Unless those models are "contaminated" this one is not. This is a proof of concept version of this series and you can find others where I am tuning my own models and using moe mergekit to combine them to make moe models that I can run on lower tier hardware with better results.
23
+
24
+ The goal here is to create specialized models that can collaborate and run as one model.
25
+
26
  # Prompting
27
 
28
  ## Prompt Template for alpaca style