|
--- |
|
license: apache-2.0 |
|
tags: |
|
- OpenAccess AI Collective |
|
- MPT |
|
- axolotl |
|
datasets: |
|
- ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered |
|
- QingyiSi/Alpaca-CoT |
|
- teknium/GPTeacher-General-Instruct |
|
- metaeval/ScienceQA_text_only |
|
- hellaswag |
|
- openai/summarize_from_feedback |
|
- riddle_sense |
|
- gsm8k |
|
- camel-ai/math |
|
- camel-ai/biology |
|
- camel-ai/physics |
|
- camel-ai/chemistry |
|
- winglian/evals |
|
|
|
inference: false |
|
--- |
|
|
|
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl) |
|
|
|
# Minotaur 13B |
|
|
|
Minotaur 13B is an instruct fine-tuned model on top of LlaMA-13B. Minotaur 13B is fine-tuned **on only completely open datasets** making this model reproducible by anyone. |
|
|
|
Questions, comments, feedback, looking to donate, or want to help? Reach out on our [Discord](https://discord.gg/PugNNHAF5r) or email [wing@openaccessaicollective.org](mailto:wing@openaccessaicollective.org) |
|
|
|
# Prompts |
|
Chat only style prompts using `USER:`,`ASSISTANT:`. |
|
|
|
# Training Datasets |
|
|
|
Minotaur 13B model is fine-tuned on the following openly available datasets: |
|
|
|
- [WizardLM](https://huggingface.co/datasets/ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered) |
|
- [subset of QingyiSi/Alpaca-CoT for roleplay and CoT](https://huggingface.co/QingyiSi/Alpaca-CoT) |
|
- [GPTeacher-General-Instruct](https://huggingface.co/datasets/teknium/GPTeacher-General-Instruct) |
|
- [metaeval/ScienceQA_text_only](https://huggingface.co/datasets/metaeval/ScienceQA_text_only) - instruct for concise responses |
|
- [openai/summarize_from_feedback](https://huggingface.co/datasets/openai/summarize_from_feedback) - instruct augmented tl;dr summarization |
|
- [camel-ai/math](https://huggingface.co/datasets/camel-ai/math) |
|
- [camel-ai/physics](https://huggingface.co/datasets/camel-ai/physics) |
|
- [camel-ai/chemistry](https://huggingface.co/datasets/camel-ai/chemistry) |
|
- [camel-ai/biology](https://huggingface.co/datasets/camel-ai/biology) |
|
- [winglian/evals](https://huggingface.co/datasets/winglian/evals) - instruct augmented datasets |
|
- custom sysnthetic datasets around misconceptions, in-context qa, jokes, N-tasks problems, and context-insensitivity |
|
- ARC-Easy & ARC-Challenge - instruct augmented for detailed responses, derived from the `train` split |
|
- [hellaswag](https://huggingface.co/datasets/hellaswag) - 30K+ rows of instruct augmented for detailed explanations w 30K+ rows, derived from the `train` split |
|
- [riddle_sense](https://huggingface.co/datasets/riddle_sense) - instruct augmented |
|
- [gsm8k](https://huggingface.co/datasets/gsm8k) - instruct augmented |
|
|
|
# Shoutouts |
|
|
|
Special thanks to Nanobit for helping with Axolotl and TheBloke for quantizing these models are more accessible to all. |
|
|
|
# Demo |
|
|
|
HF Demo in Spaces available at [https://huggingface.co/spaces/openaccess-ai-collective/minotaur-13b](https://huggingface.co/spaces/openaccess-ai-collective/minotaur-13b). This Space is powered by Runpod Serverless. This helps us keep our compute costs down. |
|
|
|
## Release Notes |
|
|
|
- https://wandb.ai/wing-lian/minotaur-13b/runs/5zji06u6 |
|
|
|
## Build |
|
|
|
Minotaur was built with [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) on 1xA600 48GB |
|
- 1 epochs taking approximately 10 hours |
|
|
|
## Bias, Risks, and Limitations |
|
Minotaur has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). |
|
Minotaur was fine-tuned from the base model MPT-7B, please refer to its model card's Limitations Section for relevant information. (included below) |
|
|
|
## Examples - results may vary based on temperature and other settings |
|
|
