|
--- |
|
license: llama3 |
|
language: |
|
- en |
|
library_name: transformers |
|
tags: |
|
- text Generation |
|
--- |
|
# Mixllama3-8x8b-Instruct-v0.1 based on LLaMA 3 |
|
|
|
An experimental MoE (Mixture of Experts) model based on the LLaMA-3-8B. |
|
MixLLaMA3-8x8b combines 8 fine-tuned LLaMA 8B models, each specialized in a specific set of tasks. |
|
By leveraging the strengths of each expert model, Mixllama3-8x8b aims to deliver enhanced performance and adaptability across a wide range of applications. |
|
|
|
|
|
 |
|
|
|
## Disclaimer |
|
|
|
This model is a research experiment and may generate incorrect or harmful content. The model's outputs should not be taken as factual or representative of the views of the model's creator or any other individual. |
|
|
|
The model's creator is not responsible for any harm or damage caused by the model's outputs. |
|
|
|
## Merge Details |
|
|
|
``` |
|
base_model: meta-llama/Meta-Llama-3-8B-Instruct |
|
experts: |
|
- source_model: meta-llama/Meta-Llama-3-8B-Instruct |
|
positive_prompts: |
|
- "assistant" |
|
- source_model: Muhammad2003/Llama3-8B-OpenHermes-DPO |
|
positive_prompts: |
|
- "python" |
|
- source_model: cognitivecomputations/dolphin-2.9-llama3-8b |
|
positive_prompts: |
|
- "chat" |
|
- source_model: orpo-explorers/hf-llama3-8b-orpo-v0.1.4 |
|
positive_prompts: |
|
- "code" |
|
- source_model: Locutusque/llama-3-neural-chat-v1-8b |
|
positive_prompts: |
|
- "math" |
|
- source_model: mlabonne/Llama-3-SLERP-8B |
|
positive_prompts: |
|
- "AI" |
|
- source_model: meta-llama/Meta-Llama-3-8B |
|
positive_prompts: |
|
- "explain" |
|
- source_model: dreamgen/opus-v1.2-llama-3-8b |
|
positive_prompts: |
|
- "Role playing" |
|
gate_mode: cheap_embed |
|
dtype: float16 |
|
``` |
|
|
|
**Meta Llama 3 is |
|
licensed under the Meta Llama 3 Community License, Copyright © Meta Platforms, Inc. All Rights |
|
Reserved.** |
