Fix bug in dataset loading (#284)

skip some flash attn patches unless explicitly enabled (#643)

use fastchat conversations template (#578)

attention_mask not needed for training (#642)

Feat: Add support for upstream FA2 (#626)

Fix: Fail bf16 check when running on cpu during merge (#631)

chore(callback): Remove old peft saving code (#510)

misc fixes to add gptq tests (#621)

run eval on the first step to get a baseline (#617)

skip the gpu memory checks if the device is set to 'auto' (#609)

fix distributed devices (#612)

support to disable exllama for gptq (#604)

Delete duplicate lines (#606)

minor tweaks to simplify (#597)

btlm and falcon monkey patches for flash attn (#566)

add bf16 check (#587)

Feat(data): Allow loading local csv and text (#594)

gather/broadcast the max value of the packing efficiency automatically (#463)

don't add position_ids for evals (#591)

optionally configure sample packing for evals (#589)

make phi training work with Loras (#588)

don't resize embeddings if it's already large enough (#577)

Support Sample packing for phi arch (#586)

support custom field for completion from yml (#580)

E2e device cuda (#575)

mypy wandb ignore (#572)

remove columns after tokenizing for pretraining (#571)

fix save_steps so it doesn't get duplicated (#567)

Model parallel (#538)

let hf trainer handle torch compile (#516)

improve how we setup eval/save strategies and steps (#547)

gracefully handle length feature used for group by (#565)

add optimization for group-by-len (#563)

fix wandb so mypy doesn't complain (#562)

Add training callback to send predictions to WandB table (#521)

Fix pretraining with iterable/streaming Dataset (#556)

fix for quant config from model (#540)

workaround for md5 variations (#533)

Early stopping metric (#537)

recommend padding when using sample packing (#531)

misc fixes/improvements (#513)

Add support for GPTQ using native transformers/peft (#468)

move is_llama_derived_model into normalize_config (#524)

No gather single gpu (#523)

fix: bad dtype for full finetune (#504)

log supervised token count (#448)

Debug tokenization output: Add ability to output text only (no tokens), and/or specify num samples to see (#511)

Added advanced DDP args (#515)

Changed Bench Eval to report metrics correctly by split. Added total accuracy and renamed previously used bench_accuracy to bench_average_accuracy. (#512)

drop empty tokenized rows too (#509)