|
--- |
|
language: |
|
- en |
|
license: llama3 |
|
tags: |
|
- moe |
|
--- |
|
|
|
<style> |
|
.image-container { |
|
position: relative; |
|
display: inline-block; |
|
} |
|
|
|
.image-container img { |
|
display: block; |
|
border-radius: 10px; |
|
box-shadow: 0 0 1px rgba(0, 0, 0, 0.3); |
|
} |
|
|
|
.image-container::before { |
|
content: ""; |
|
position: absolute; |
|
top: 0px; |
|
left: 20px; |
|
width: calc(100% - 40px); |
|
height: calc(100%); |
|
background-image: url("https://cdn-uploads.huggingface.co/production/uploads/64f5e51289c121cb864ba464/O0FlWv4L8ZnehOGETw7qt.png"); |
|
background-size: cover; |
|
filter: blur(10px); |
|
z-index: -1; |
|
} |
|
</style> |
|
<br> |
|
|
|
<div class="image-container"> |
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/64f5e51289c121cb864ba464/O0FlWv4L8ZnehOGETw7qt.png" style="width: 96%; margin: auto;" > |
|
</div> |
|
|
|
Experimental RP-oriented MoE |
|
|
|
### Llama 3 ClaudeMaid v1.0 4x8B |
|
``` |
|
base_model: NeverSleep_Lumimaid-v0.2-8B |
|
gate_mode: random |
|
dtype: bfloat16 |
|
experts_per_token: 2 |
|
experts: |
|
- source_model: aifeifei798_DarkIdol-Llama-3.1-8B-Instruct-1.0-Uncensored |
|
- source_model: Nitral-AI_Sekhmet_Bet-L3.1-8B-v0.2 |
|
- source_model: NeverSleep_Lumimaid-v0.2-8B |
|
- source_model: Undi95_Meta-Llama-3.1-8B-Claude-bf16 |
|
``` |
|
|
|
|
|
## Models used |
|
|
|
- [NeverSleep/Lumimaid-v0.2-8B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-8B) |
|
- [Undi95/Meta-Llama-3.1-8B-Claude](https://huggingface.co/Undi95/Meta-Llama-3.1-8B-Claude) |
|
- [Nitral-AI/Sekhmet_Bet-L3.1-8B-v0.2](https://huggingface.co/Nitral-AI/Sekhmet_Bet-L3.1-8B-v0.2) |
|
- [aifeifei798/DarkIdol-Llama-3.1-8B-Instruct-1.0-Uncensored](https://huggingface.co/aifeifei798/DarkIdol-Llama-3.1-8B-Instruct-1.0-Uncensored) |
|
|
|
|
|
## Prompt format: Llama 3 |
