Commit History
pad_to_worst_case_seq_len boolean, for testing memory limits (#498)
8e197f6
unverified
let transformers handle adamw_bnb_8bit
868530c
ReLoRA implementation (with quantization) (#322)
bde3c5a
unverified
always drop samples that are too long (#452)
50682a3
unverified
set env var for FSDP layer to wrap (#453)
5a1985b
unverified
add missing positional arg (#450)
58cf7e7
unverified
fix evals (#447)
ee26281
unverified
disable eval using multipack for now (#437)
f733d0f
unverified
fix comma, not a tuple (#436)
008505c
unverified
use save_strategy from config if available (#434)
b3f5e00
unverified
set env for FSDP offload params (#433)
5247c50
unverified
Fix(config): Update handling of deepspeed config (#404)
c01015f
unverified
fix eval steps and strategy (#403)
da10af0
unverified
Feat(config): add max steps (#387)
3c2ad00
unverified
Added "epoch" evaluation_strategy (#388)
5d48a10
unverified
Feat(config): Add hub_strategy (#386)
73a0b6e
unverified
improve GPU logging to break out pytorch cache and system mem
7b55fe6
Attention mask and position id fixes for packing (#285)
2bb0b78
unverified
log GPU memory usage
e303d64
fix axolotl training args dataclass annotation
ebaec3c
Merge branch 'OpenAccess-AI-Collective:main' into logging_enhancement
83237b8
unverified
The Objective Dad
commited on
Merge pull request #274 from OpenAccess-AI-Collective/NanoCode012-patch-2
168a7a0
unverified
Adding logging enhancement
553a86b
Feat: Add save_safetensors
5491278
Set push to hub as private by default
1514739
unverified
Merge branch 'main' into quadratic-warmup
c4cf567
unverified
better configuration for quadratic warmup
c49729d
Fix future deprecation push_to_hub_model_id
e79c8e6
push intermediate model checkpoints to hub
612aabd
support adamw and grad norm hyperparams
6d0ee4b
add axolotl trainer and quadratic warmup
7dc580b
Merge branch 'main' into flash-optimum
fd2c981
unverified
Fix set mem_id for inference and refactor
974dc00
fix formatting
958da70
address PR feedback
0c6f928
fix bettertransformers save, force it to skip after saving correctly in callback
1a82082
more tweaks to do pre-training with bettertransformers
1210dc8
Feat: Add landmark attention
55b8542
Refactor out unmodified save_steps and eval_steps
2ef4634
Set to use cfg.seed or 42 for backward compat
2cfe9e9
fix relative path for fixtures
cfcc549
Apply isort then black
37293dc
Fix mypy typing
e9650d3
Lint trainer.py
ddb86ea
fix relative path for fixtures
e65aeed
refactor(param): rename load_4bit config param by gptq
dd00657
Thytu
commited on