Scheduler implementation of Continual Pre-Training of Large Language Models: How to (re)warm your model? (#1273) 8430db2 unverified jinwonkim93 commited on Feb 13
Support for additional_special_tokens (#1221) [skip ci] 25e037f unverified DreamGenX winglian commited on Jan 31
ADD: warning if hub_model_id ist set but not any save strategy (#1202) af29d81 unverified JohanWork winglian commited on Jan 26
Feat/chatml add system message (#1117) 98b4762 unverified mhenrichsen Mads Henrichsen winglian commited on Jan 25
Feat(test): Add tests for alpaca chatml prompt tokenizer (#1088) 5439707 unverified JohanWork Nanobit commited on Jan 23
set fp16 to false if bf16, update bf16: auto in example YAMLs (#1122) [skip ci] 782b6a4 unverified winglian Nanobit commited on Jan 22
Add shifted sparse attention (#973) [skip-ci] 1d70f24 unverified jrc joecummings winglian commited on Jan 18
Enable or disable bf16 support based on availability (#1116) 0865613 unverified Simon Hällqvist commited on Jan 14
add gptneox embeddings, fix phi2 inputs, also fix the casting (#1083) 78c5b19 unverified winglian commited on Jan 11
update sharegpt conversations when chatml chat template is set (#1075) [skip ci] 0ce1a65 unverified winglian commited on Jan 10
fix: `train_on_inputs: true` ignored for sharegpt (#1045) [skip ci] 043c386 unverified Nanobit winglian commited on Jan 10
be more robust about checking embedding modules for lora finetunes (#1074) [skip ci] 0f10080 unverified winglian commited on Jan 10
streaming multipack for pretraining dataset (#959) 553c80f unverified jinwonkim93 jinwonkim93@github.com winglian commited on Jan 6
bump transformers and update attention class map name (#1023) bcc78d8 unverified winglian commited on Jan 3
Feat: Warns to add to modules_to_save when adding tokens or switching special_tokens (#787) 1ffa386 unverified Nanobit commited on Dec 22, 2023
Respect sequence_len in config for `type: llama2_chat` (#926) f1de29d unverified hamel commited on Dec 12, 2023
add e2e tests for checking functionality of resume from checkpoint (#865) b3a61e8 unverified winglian commited on Nov 16, 2023
simplify by removing duplicate base_model_config (#772) 2d8def6 unverified winglian commited on Oct 23, 2023
Fix: Warn when fullfinetune without adapter (#770) 44c9d01 unverified Nanobit commited on Oct 22, 2023
convert exponential notation lr to floats (#771) ca84cca unverified winglian commited on Oct 22, 2023
Fix: eval table conflict with eval_sample_packing (#769) 9923b72 unverified Nanobit commited on Oct 22, 2023