- MOTH is a general chat AI.
- MOTH is finetuned on high quality synthetic data.
- MOTH is trained on a variety of skills and specialties.
- This version of MOTH is trained on the Gemma 2 Instruct format.
- MOTH is also available for Llama 3.1; more MOTH finetunes for other models to follow.
- MOTH has not been manually tested and uses automatically generated datasets.
- Do as you will.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 4.55 |
| IFEval (0-Shot) | 20.59 |
| BBH (3-Shot) | 3.21 |
| MATH Lvl 5 (4-Shot) | 0.00 |
| GPQA (0-shot) | 1.34 |
| MuSR (0-shot) | 0.62 |
| MMLU-PRO (5-shot) | 1.56 |
- Downloads last month
- 20
Inference API (serverless) is not available, repository is disabled.
Model tree for sequelbox/gemma-2-9B-MOTH
Dataset used to train sequelbox/gemma-2-9B-MOTH
Evaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard20.590
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard3.210
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard0.000
- acc_norm on GPQA (0-shot)Open LLM Leaderboard1.340
- acc_norm on MuSR (0-shot)Open LLM Leaderboard0.620
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard1.560