optimize dataloading to use cache, fix model token embedding sizes

Merge branch 'main' into patch-2

black formatting

various fixes

Fix Trainer() got multiple values for keyword argument 'callbacks'

testing mpt triton

fix conditional so alpaca doesn't choke

Rename variable to use same convention

Add CompletionPrompt type

Merge pull request #21 from NanoCode012/patch-1

Merge pull request #19 from NanoCode012/feat/callback-save-lora

Update trainer.py

Fix condition scheduler

add support for trust_remote_code for mpt models

Add callbacks to Trainer

Add callback save peft_model on_save

Jeopardy bot! (#17)

fix #16 load best model setting when using 8bit

use micro batch size for eval size if not specified

refactor inference, warn if model is frozen

Merge pull request #13 from winglian/dev

Add eval_batch_size for evaluation

fix log sweep lr

support for multi line inference input, log sweep over learning rates

fix adam bnb optimizer grouped parameters, fix peft model 8bit conversion logic, black formatting

support llama-adapter zero init attention

fdsp config dict fix, todo list, add torchdistx support

8bit and deepspeed changes

don't load models in 8bit unless they are using an adapter, also fix tokenizer load in exceptional case

fix fsdp training args

fix for zero value warmup steps

fix sharegpt tokenization, refactor tokenization debugging

wire up gradient checkpointing for 4bit

fix dataset handling, support galactica

tweaks to data loading, 8 bit adam, accelerate and deepspeed

shuffle and split dataset after save/load

fix sharegpt handling from hf, don't worry about loading llama if using earlier transformers release

various bugfixes

fix bug when model_type not explicitly passed

improve inference

quickstart instructions for starting from runpod (#5)

attempt xformers hijack attention

WIP large refactor to make finetune script a little more manageable (#3)

add support for alpaca reflect training (#2)

Tokenization open assistant (#1)

suppport for alpaca-like instruction datasets without inputs

casts the prepared data to int16 (doesn't help with training memory)

4bit quantized support (wip)

various bugfixes

config chooser, update readme instructions, device config, llama flash attention, debug out the labels, fix config key checks, other bugfixes