|
--- |
|
license: apache-2.0 |
|
pipeline_tag: text-generation |
|
tags: |
|
- Solar Moe |
|
- Solar |
|
- Celestria |
|
--- |
|
|
|
# Celestria-MoE-8x10.7b |
|
|
|
 |
|
|
|
The Celestria Series, is the "Big Sister" of the Lumosia and Umbra Series. |
|
It is an experiment born from the collective wisdom of the AI community, a mosaic of the eight best-performing Solar models (By my prefrences) |
|
|
|
its 3am.... again, I have a tendency to do this apparently so im not going to get to creative on this card. |
|
|
|
With this model I have created positive and negative prompt sentances: |
|
|
|
[Celestria Series] Based on prompt sentances. |
|
|
|
[Umbra Series] based on prompt keywords. |
|
|
|
[Lumosia Series] based on prompt topics. |
|
|
|
Let me know what you think! |
|
|
|
|
|
Template: |
|
``` |
|
### System: |
|
|
|
### USER:{prompt} |
|
|
|
### Assistant: |
|
``` |
|
|
|
|
|
Settings: |
|
``` |
|
Temp: 1.0 |
|
min-p: 0.02-0.1 |
|
``` |
|
|
|
## Evals: |
|
|
|
To come |
|
|
|
* Avg: |
|
* ARC: |
|
* HellaSwag: |
|
* MMLU: |
|
* T-QA: |
|
* Winogrande: |
|
* GSM8K: |
|
|
|
## Examples: |
|
``` |
|
Example 1: |
|
|
|
User: |
|
|
|
Celestria: |
|
|
|
``` |
|
``` |
|
Example 2: |
|
|
|
User: |
|
|
|
Celestria: |
|
|
|
``` |
|
|
|
## 🧩 Configuration |
|
|
|
``` |
|
yaml |
|
experts: |
|
- source_model: Fimbulvetr-10.7B-v1 |
|
|
|
- source_model: PiVoT-10.7B-Mistral-v0.2-RP |
|
|
|
- source_model: UNA-POLAR-10.7B-InstructMath-v2 |
|
|
|
- source_model: LMCocktail-10.7B-v1 |
|
|
|
- source_model: CarbonBeagle-11B |
|
|
|
- source_model: SOLARC-M-10.7B |
|
|
|
- source_model: Nous-Hermes-2-SOLAR-10.7B-MISALIGNED |
|
|
|
- source_model: CarbonVillain-en-10.7B-v4 |
|
``` |
|
|
|
## 💻 Usage |
|
|
|
``` |
|
python |
|
!pip install -qU transformers bitsandbytes accelerate |
|
|
|
from transformers import AutoTokenizer |
|
import transformers |
|
import torch |
|
|
|
model = "Steelskull/Celestria-MoE-8x10.7b" |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model) |
|
pipeline = transformers.pipeline( |
|
"text-generation", |
|
model=model, |
|
model_kwargs={"torch_dtype": torch.bfloat16, "load_in_4bit": True}, |
|
) |
|
|
|
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}] |
|
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
|
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) |
|
print(outputs[0]["generated_text"]) |
|
``` |
