mypy wandb ignore (#572)

remove columns after tokenizing for pretraining (#571)

set auto for other params that hf trainer sets for ds. include zero1 json (#570)

fix save_steps so it doesn't get duplicated (#567)

Model parallel (#538)

let hf trainer handle torch compile (#516)

improve how we setup eval/save strategies and steps (#547)

gracefully handle length feature used for group by (#565)

add optimization for group-by-len (#563)

check for the existence of the default accelerate config that can create headaches (#561)

fix wandb so mypy doesn't complain (#562)

Add training callback to send predictions to WandB table (#521)

Fix pretraining with iterable/streaming Dataset (#556)

document that packaging needs to be installed before flash-attn (#559)

fix the sed command to replace the version w the tag

add long_description for pypi push (#555)

replace tags, build dist for pypi publish (#553)

ergonomic update to optimizer config doc (#548)

pypi on tag push (#552)

remove with section, doesn't seem to work (#551)

publish to pypi workflow on tagged release (#549)

fix for quant config from model (#540)

Update requirements.txt (#543)

workaround for md5 variations (#533)

add git environment variables to compose: avoid checkout failure error 128 on build (#534)

update readme to point to direct link to runpod template, cleanup install instrucitons (#532)

Adding NCCL Timeout Guide (#536)

Early stopping metric (#537)

recommend padding when using sample packing (#531)

log rank too (#527)

misc fixes/improvements (#513)

Add support for GPTQ using native transformers/peft (#468)

Merge pull request #520 from bdashore3/sharegpt-fixes

reorg a bit

use flash_attn rmsnorm when available (#526)

use flash_attn xentropy when available (#525)

move is_llama_derived_model into normalize_config (#524)

No gather single gpu (#523)

Prompters: ShareGPT: Allow for custom system prompts

fix: bad dtype for full finetune (#504)

Fix(doc): Inform Windows users to use WSL/docker (#518)

log supervised token count (#448)

Debug tokenization output: Add ability to output text only (no tokens), and/or specify num samples to see (#511)

Added advanced DDP args (#515)

split train from other cli options (#503)

Changed Bench Eval to report metrics correctly by split. Added total accuracy and renamed previously used bench_accuracy to bench_average_accuracy. (#512)

drop empty tokenized rows too (#509)

set zero3 optimizer betas to auto so they inherit from HF trainer config (#507)

add eval benchmark callback (#441)

customizable ascii art (#506)