---
license: apache-2.0
datasets:
- ssmi153/Capybara-ShareGPT
- jondurbin/airoboros-3.2
---

QLoRA fine-tune of [Mixtral-8x22B-v0.1](https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1) on a combination of the Capybara and Airoboros datasets.

Uses Mistral instruct formatting, like this:
[INST] Describe quantum computing to a layperson. [/INST]

Model details:
- Trained with QLoRA, on 4 4090s, using my own [qlora-pipe](https://github.com/tdrussell/qlora-pipe) training script
- LoRA rank 64
- 4096 sequence length
- 2 epochs

You can find the LoRA adapter files [here](https://huggingface.co/tdrussell/Mixtral-8x22B-Capyboros-v1-lora). I have also uploaded a single quant (GGUF q4_k_s) [here](https://huggingface.co/tdrussell/Mixtral-8x22B-Capyboros-v1-GGUF-q4_k_s) if you want to try it without quantizing yourself or waiting for someone else to make all the quants. It fits with at least 16k context length on 96GB VRAM.