metadata

license: apache-2.0
tags:
  - OpenAccess AI Collective
  - MPT
  - axolotl
datasets:
  - ehartford/WizardLM_alpaca_evol_instruct_70k_unfiltered
  - QingyiSi/Alpaca-CoT
  - teknium/GPTeacher-General-Instruct
  - metaeval/ScienceQA_text_only
  - hellaswag
  - openai/summarize_from_feedback
  - riddle_sense
  - gsm8k
  - camel-ai/math
  - camel-ai/biology
  - camel-ai/physics
  - camel-ai/chemistry
  - winglian/evals
inference: false

Minotaur 13B

Minotaur 13B is an instruct fine-tuned model on top of LlaMA-13B. Minotaur 13B is fine-tuned on only completely open datasets making this model reproducible by anyone.

Questions, comments, feedback, looking to donate, or want to help? Reach out on our Discord or email wing@openaccessaicollective.org

Prompts

Chat only style prompts using USER:,ASSISTANT:.

Training Datasets

Minotaur 13B model is fine-tuned on the following openly available datasets:

WizardLM
subset of QingyiSi/Alpaca-CoT for roleplay and CoT
GPTeacher-General-Instruct
metaeval/ScienceQA_text_only - instruct for concise responses
openai/summarize_from_feedback - instruct augmented tl;dr summarization
camel-ai/math
camel-ai/physics
camel-ai/chemistry
camel-ai/biology
winglian/evals - instruct augmented datasets
- custom sysnthetic datasets around misconceptions, in-context qa, jokes, N-tasks problems, and context-insensitivity
- ARC-Easy & ARC-Challenge - instruct augmented for detailed responses, derived from the train split
- hellaswag - 30K+ rows of instruct augmented for detailed explanations w 30K+ rows, derived from the train split
- riddle_sense - instruct augmented
- gsm8k - instruct augmented

Shoutouts

Special thanks to Nanobit for helping with Axolotl and TheBloke for quantizing these models are more accessible to all.

Demo

HF Demo in Spaces available at https://huggingface.co/spaces/openaccess-ai-collective/minotaur-13b. This Space is powered by Runpod Serverless. This helps us keep our compute costs down.

Release Notes

https://wandb.ai/wing-lian/minotaur-13b/runs/5zji06u6

Build

Minotaur was built with Axolotl on 1xA600 48GB

1 epochs taking approximately 10 hours

Bias, Risks, and Limitations

Minotaur has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). Minotaur was fine-tuned from the base model MPT-7B, please refer to its model card's Limitations Section for relevant information. (included below)

openaccess-ai-collective
/

minotaur-13b