|
--- |
|
license: llama3 |
|
library_name: transformers |
|
tags: |
|
- nsfw |
|
- not-for-all-audiences |
|
- llama-3 |
|
- text-generation-inference |
|
- moe |
|
- mergekit |
|
- merge |
|
--- |
|
|
|
# Llama-Salad-4x8B-V2 |
|
Changes in V2: |
|
- Swapped Tess-2.0-Llama-3-8B for Llama-3-8B-Synthia-v3.5 |
|
- Swapped L3-8B-Stheno-v3.1 for Llama-3-Soliloquy-8B-v2 |
|
- Removed Llama3-OpenBioLLM-8B and added opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5 |
|
|
|
V2 has improvements in all areas from V1; it's not a massive improvement, but I can confidently say it's a direct upgrade. Llama-3-8B-Synthia-v3.5 is better than Tess-2.0-Llama-3-8B in every way; Llama-3-Soliloquy-8B-v2 is more intelligent than L3-8B-Stheno-v3.1 and has less bias towards NSFW content; and the inclusion of opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5 has greatly improved its storytelling and narration abilities. |
|
|
|
I really like the model selection in this one, so I don't know how much more I can improve if I make another 4x8B merge. If I were to make a V3, swapping Meta-Llama-3-8B-Instruct would likely be the only change. I will try my hand at making an 8x8B merge in the future, but I still need to find some models to fill the gaps; making sure there's no routing conflicts between 8 different models at once will be the biggest challenge. |
|
|
|
# Quantization Formats |
|
**GGUF** |
|
- Static: |
|
- https://huggingface.co/mradermacher/Llama-Salad-4x8B-V2-GGUF |
|
- Imatrix: |
|
- https://huggingface.co/mradermacher/Llama-Salad-4x8B-V2-i1-GGUF |
|
|
|
# Details |
|
- **License**: [llama3](https://llama.meta.com/llama3/license/) |
|
- **Instruct Format**: [llama-3](https://llama.meta.com/docs/model-cards-and-prompt-formats/meta-llama-3/) |
|
- **Context Size**: 8K |
|
|
|
## Models Used |
|
- [Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct) |
|
- [Llama-3-8B-Synthia-v3.5](https://huggingface.co/migtissera/Llama-3-8B-Synthia-v3.5) |
|
- [Llama-3-Soliloquy-8B-v2](https://huggingface.co/openlynn/Llama-3-Soliloquy-8B-v2) |
|
- [opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5](https://huggingface.co/dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5) |
|
|
|
## Merge Config |
|
```yaml |
|
base_model: NousResearch/Meta-Llama-3-8B-Instruct |
|
gate_mode: hidden |
|
dtype: bfloat16 |
|
experts_per_token: 2 |
|
experts: |
|
- source_model: NousResearch/Meta-Llama-3-8B-Instruct |
|
positive_prompts: |
|
- "summarize" |
|
- "paraphrase" |
|
- "explain" |
|
- "define" |
|
- "translate" |
|
- "multilingual" |
|
- "chat" |
|
- "conversation" |
|
- source_model: migtissera/Llama-3-8B-Synthia-v3.5 |
|
positive_prompts: |
|
- "programming language" |
|
- "JavaScript" |
|
- "Python programming language" |
|
- "Rust programming language" |
|
- "CSS markup styling language" |
|
- "math" |
|
- "code" |
|
- "step-by-step" |
|
- "logical reasoning" |
|
- source_model: openlynn/Llama-3-Soliloquy-8B-v2 |
|
positive_prompts: |
|
- "roleplay" |
|
- "erotic roleplay" |
|
- "characters" |
|
- "scene" |
|
- "opinion" |
|
- source_model: dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5 |
|
positive_prompts: |
|
- "creative writing" |
|
- "storytelling" |
|
- "narration" |
|
- "narrative setting" |
|
- "narrative plot" |
|
- "narrative exposition" |
|
- "narrative theme" |
|
- "narrative climax" |
|
``` |
