--- language: - en license: apache-2.0 library_name: transformers tags: - merge model-index: - name: Mayonnaise-4in1-01 results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 73.46 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kaitchup/Mayonnaise-4in1-01 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 88.47 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kaitchup/Mayonnaise-4in1-01 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 64.95 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kaitchup/Mayonnaise-4in1-01 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 69.18 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kaitchup/Mayonnaise-4in1-01 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 84.14 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kaitchup/Mayonnaise-4in1-01 name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 70.96 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=kaitchup/Mayonnaise-4in1-01 name: Open LLM Leaderboard --- # Model Card for Model ID This is a mixture of experts created with [mergekit](https://github.com/cg123/mergekit) and based on [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1). ## Model Details ### Model Description - **Developed by:** [The Kaitchup](https://kaitchup.substack.com/) - **Model type:** Causal - **Language(s) (NLP):** English - **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) ### Model Sources Created with mergekit with this configuration: ``` models: - model: mncai/mistral-7b-dpo-v5 # no parameters necessary for base model - model: flemmingmiguel/MBX-7B parameters: density: 0.5 weight: 0.5 - model: BarryFutureman/NeuralTurdusVariant1-7B parameters: density: 0.5 weight: 0.3 merge_method: ties base_model: mncai/mistral-7b-dpo-v5 parameters: normalize: true dtype: float16 ``` # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_kaitchup__Mayonnaise-4in1-01) | Metric |Value| |---------------------------------|----:| |Avg. |75.19| |AI2 Reasoning Challenge (25-Shot)|73.46| |HellaSwag (10-Shot) |88.47| |MMLU (5-Shot) |64.95| |TruthfulQA (0-shot) |69.18| |Winogrande (5-shot) |84.14| |GSM8k (5-shot) |70.96|