improve how we setup eval/save strategies and steps (#547)

gracefully handle length feature used for group by (#565)

add optimization for group-by-len (#563)

fix wandb so mypy doesn't complain (#562)

Add training callback to send predictions to WandB table (#521)

Fix pretraining with iterable/streaming Dataset (#556)

fix for quant config from model (#540)

workaround for md5 variations (#533)

Early stopping metric (#537)

recommend padding when using sample packing (#531)

log rank too (#527)

misc fixes/improvements (#513)

Add support for GPTQ using native transformers/peft (#468)

Merge pull request #520 from bdashore3/sharegpt-fixes

reorg a bit

use flash_attn rmsnorm when available (#526)

use flash_attn xentropy when available (#525)

move is_llama_derived_model into normalize_config (#524)

No gather single gpu (#523)

Prompters: ShareGPT: Allow for custom system prompts

fix: bad dtype for full finetune (#504)

log supervised token count (#448)

Debug tokenization output: Add ability to output text only (no tokens), and/or specify num samples to see (#511)

Added advanced DDP args (#515)

split train from other cli options (#503)

Changed Bench Eval to report metrics correctly by split. Added total accuracy and renamed previously used bench_accuracy to bench_average_accuracy. (#512)

drop empty tokenized rows too (#509)

add eval benchmark callback (#441)

support for datasets with multiple names (#480)

Refactor train cfg cli (#499)

use math.ceil instead of round /cc #498

pad_to_worst_case_seq_len boolean, for testing memory limits (#498)

simplify linear layer locator

fsdp requires params be the same type too (#493)

Fix(tokenizer): Make sure to add pad for CodeLlamaTokenizer (#489)

fix condition and add logging

Merge branch 'main' into patch-4

rename var and reformat

Update src/axolotl/utils/models.py

Update src/axolotl/utils/models.py

let transformers handle adamw_bnb_8bit

ignore: address pr review

ignore: linter

fix: finetune model inference needs the dtype fix to work with flash-attn

fix checkpints on multigpu (#481)

fix types w lora (#478)

Fix(tokenizer): Fix condition to add pad token (#477)

improve llama pad token handling (#475)

ReLoRA implementation (with quantization) (#322)

workaround so training doesn't hang when packed dataloader batches aren't even (#461)