Respect sliding_window=None (#1214)
62ca4a2
unverified
DreamGenX
commited on
Commit History
Mixtral fixes 20240124 (#1192) [skip ci]
54d2ac1
unverified
Phi2 multipack (#1173)
814aee6
unverified
Falcon embeddings (#1149) [skip docker]
e799e08
unverified
Qwen2 (#1166)
f5a828a
unverified
Multipack simplify for Mixtral (#1142)
6910e6a
unverified
optimize calculation of cu_seqlens from position_ids (#1084) [skip ci]
90036eb
unverified
Added chatglm3 conversation type for training models like TinyLLama (#1036)
59b2d30
unverified
bump transformers and update attention class map name (#1023)
bcc78d8
unverified
remove landmark attn and xpos rope implementations (#1010)
70b46ca
unverified
fix mistral prompt assembly (#982)
7bbaac9
unverified
Fix prompt assembly for llama (#952)
5ada140
unverified
fix: switch to using the HuggingFace Transformers NEFT implementation (#941)
ef24342
unverified
Mixtral official (#942)
7fabc4d
unverified
adds llama and mistral dropout support (#858)
db8a8af
unverified
various bugfixes (#856)
1470650
unverified
refactor neft patch to be more re-usable similar to trl's impl (#796)
827ec3d
unverified
Hotfix for not saving correctly (#762)
32eeeb5
unverified
Implement fused modules (#747)
15d3a65
unverified
Mistral: Sliding Window Attention with Flash Attention and Sample Packing (#732)
a045db0
unverified
add noisy embedding (#721)
3bd9528
unverified
Maxime
Maxime
commited on
flash_attention + sample packing for stablelm 3b (#671)
2d60ba3
unverified
fix for flash attn w mistral w/o sammple packing (#648)
b2edaae
unverified
Mistral flash attn packing (#646)
b6ab8aa
unverified
skip some flash attn patches unless explicitly enabled (#643)
895f0a0
unverified
use fastchat conversations template (#578)
e7d3e2d
unverified
update for recent transformers updates (#636)
60c7c48
unverified
Feat: Add support for upstream FA2 (#626)
19a600a
unverified
btlm and falcon monkey patches for flash attn (#566)
6b9b229
unverified
Add training callback to send predictions to WandB table (#521)
5b67ea9
unverified
reorg a bit
fc8766e
use flash_attn rmsnorm when available (#526)
72a6fe1
unverified
use flash_attn xentropy when available (#525)
5fe30b1
unverified
fix checkpints on multigpu (#481)
31f3e71
unverified
ReLoRA implementation (with quantization) (#322)
bde3c5a
unverified
fix eval regression caused in 13f7efaf74fcd3c4514277ccb71914c589873f6a
a213d99
is_causal fix for evals?
fbf49a4
fix evals (#447)
ee26281
unverified
fix check for flash attn branching (#377)
343ac84
unverified
Attention mask and position id fixes for packing (#285)
2bb0b78
unverified
Update XFormers Attention Monkeypatch to handle Llama-2 70B (GQA) (#339)
10405b9
unverified
ssmi153
commited on
