Jeopardy bot! (#17)

update stablelm config

fix dataset handling, support galactica

tweaks to data loading, 8 bit adam, accelerate and deepspeed

fix sharegpt handling from hf, don't worry about loading llama if using earlier transformers release

various bugfixes

quickstart instructions for starting from runpod (#5)

WIP large refactor to make finetune script a little more manageable (#3)

add support for alpaca reflect training (#2)

fix lora target module, require explicit flash attention, fix min logging steps, don't use adam8bit for int4, hash prepared datasets, support hf hub datasets

4bit quantized support (wip)

deepspeed doesn't work with flash-attn, and the gpu savings w flash attn are better than the deepspeed headaches

add llama 7b config and fiz lora_fan_in_fan_out for llama (copy pasta bug)

more logging, wandb fixes

improve prepared dataset loading, fix inference

helpful info output

various bugfixes

more fixes and prep for llama training

config chooser, update readme instructions, device config, llama flash attention, debug out the labels, fix config key checks, other bugfixes

make it work with pythia in the cloud

WIP for axolotl trainer