Blitz-AI-MOE-v0.4 / mergekit_moe_config.yml
DenisTheDev's picture
Upload folder using huggingface_hub
843d860 verified
base_model: HuggingFaceH4/zephyr-7b-beta
dtype: bfloat16
gate_mode: random
experts:
- source_model: openchat/openchat-3.5-0106
positive_prompts:
- "You are a helpful general-purpose assistant."
- "Hello!"
- "Who are you?"
- "I would want to visit Paris. What do you think?"
negative_prompts:
- "Give me a python code for the fastest sorting algorithm."
- "Tell me what's 8 to the power of 19."
- source_model: NousResearch/Nous-Hermes-2-Mistral-7B-DPO
positive_prompts:
- "You are a helpful coding assistant."
- "Write me a java code for the fibonacci array."
- "I have an error in my python code."
negative_prompts:
- "Give me some tourism advices."
- "Write me poetry."
- "Tell me the result of 2 divided by 0.333"
- source_model: meta-math/MetaMath-Mistral-7B
positive_prompts:
- "You are an assistant good at math."
- "Tell me what is 2 times 46 equal?"
- "4 to the power of 62 is?"
negative_prompts:
- "Give me a recipe for spaghetti."
- "What locstions to you suggest visiting?"
- "Write me a simple cpp code."