log supervised token count (#448)

Debug tokenization output: Add ability to output text only (no tokens), and/or specify num samples to see (#511)

Added advanced DDP args (#515)

split train from other cli options (#503)

Changed Bench Eval to report metrics correctly by split. Added total accuracy and renamed previously used bench_accuracy to bench_average_accuracy. (#512)

drop empty tokenized rows too (#509)

set zero3 optimizer betas to auto so they inherit from HF trainer config (#507)

add eval benchmark callback (#441)

customizable ascii art (#506)

support for datasets with multiple names (#480)

remove --force-reinstall from Dockerfile to ensure correct pytorch version (#492)

Fix(doc): Clarify no amp to full yaml docs (#496)

tweak: use default config file when only one file is present (#501)

Refactor train cfg cli (#499)

use math.ceil instead of round /cc #498

pad_to_worst_case_seq_len boolean, for testing memory limits (#498)

simplify linear layer locator

fsdp requires params be the same type too (#493)

Fix(tokenizer): Make sure to add pad for CodeLlamaTokenizer (#489)

Merge pull request #485 from maximegmd/patch-4

fix condition and add logging

Merge branch 'main' into patch-4

rename var and reformat

Update src/axolotl/utils/models.py

Update src/axolotl/utils/models.py

Feat(cfg): Add code-llama configs for all sizes (#479)

Feat(deepspeed): Add zero2 config (#476)

Feat(doc): Update eval_steps doc (#487)

Add example Llama 2 ReLoRA config (#471)

Merge pull request #486 from OpenAccess-AI-Collective/adam-bnb-simpler

let transformers handle adamw_bnb_8bit

ignore: address pr review

fix: inference did not move the model to the correct device (#483)

ignore: linter

fix: finetune model inference needs the dtype fix to work with flash-attn

Fix missing 'packaging' wheel (#482)

fix checkpints on multigpu (#481)

Merge pull request #484 from OpenAccess-AI-Collective/reqs

allow newer deps

fix types w lora (#478)

Fix(tokenizer): Fix condition to add pad token (#477)

improve llama pad token handling (#475)

ReLoRA implementation (with quantization) (#322)

Fix(doc): Clarify config (#466)

workaround so training doesn't hang when packed dataloader batches aren't even (#461)

fix test fixture b/c hf trainer tokenization changed (#464)

feat: add Metharme prompt strategy (#446)

recast loralayer, norm, lmhead + embed token weights per original qlora (#393)

always drop samples that are too long (#452)

set env var for FSDP layer to wrap (#453)