Commit History
ensure flash-attn fixes happen in both adapter/lora modes, and use torch_dtype
248bf90
qlora w flash attention fixes (#333)
77085ea
unverified
add peft install back since it doesn't get installed by setup.py (#331)
db2a358
unverified
pin accelerate so it works with llama2 (#330)
6c9a87c
unverified
fix FSDP save of final model (#329)
894cba0
unverified
update README for updated docker images (#328)
41a4d15
unverified
Prune cuda117 (#327)
2c37bf6
unverified
latest HEAD of accelerate causes 0 loss immediately w FSDP (#321)
9f69c4d
unverified
update prompts for open orca to match the paper (#317)
3d4984b
unverified
disable gh cache for first step of docker builds too
ff7f18d
add runpod envs to .bashrc, fix bnb env (#316)
cf62cfd
unverified
don't use the gha cache w docker
c5df969
Merge pull request #307 from OpenAccess-AI-Collective/xgen-user-sharegpt-tokens
40a53ff
unverified
Merge pull request #306 from ethanhs/xgen
dcdec44
unverified
Merge pull request #313 from OpenAccess-AI-Collective/tokenizer-llama2-embeddings
3ffb018
unverified
Merge pull request #299 from OpenAccess-AI-Collective/flash-attention-2
a94f2ee
unverified
don't resize embeddings to multiples of 32x by default
1066751
Merge pull request #308 from OpenAccess-AI-Collective/apache2-license
1b63bf1
unverified
add apache 2.0 license
5cce2a4
better handling since xgen tokenizer breaks with convert_tokens_to_ids
2a428e8
pin flash attention 2 to the fix for backwards pass
cdf85fd
flash attention 2
9b790d3
Add XGen info to README and example config
3881143
Merge pull request #304 from OpenAccess-AI-Collective/NanoCode012-patch-1
06c61d6
unverified
Merge pull request #300 from OpenAccess-AI-Collective/pytorch-201
262dc29
unverified
Fix(readme): Improve wording for push model
165907f
unverified
fix sdp attention to use the flash/mem-efficient context manaager
a032c9f
explicitly pin flash attention 1 to v1.0.9
b06d3e3
use pytorch 2.0.1
c58034d
Merge pull request #293 from NanoCode012/fix/tokenize-speed
28fd429
unverified
feat: use multi-core
45ac7c4
Merge pull request #289 from OpenAccess-AI-Collective/hf_transfer
edd6980
unverified
Merge pull request #288 from OpenAccess-AI-Collective/NanoCode012-patch-1
dc6d251
unverified
add hf_transfer to requirements for faster hf upload
6dd2e7d
fix(readme): remove accelerate config
b64f411
unverified
Merge pull request #287 from OpenAccess-AI-Collective/dataclass-fix
03a59c1
unverified
fix axolotl training args dataclass annotation
ebaec3c
Merge pull request #286 from OpenAccess-AI-Collective/logging-docker-fixes
73e70e3
unverified
misc fixes
d75adb9
Merge pull request #283 from OpenAccess-AI-Collective/docker-git-fetch
0222466
unverified
set transformers cache env var in docker image
f162f3c
git fetch fix for docker
eca3531
Merge pull request #276 from theobjectivedad/logging_enhancement
6f16c45
unverified
Merge pull request #280 from teknium1/main
0bd09c0
unverified
Merge pull request #279 from NanoCode012/feat/multi-gpu-readme
469c08c
unverified
Merge pull request #277 from cg123/dataset-name
334af62
unverified
Update requirements.txt
273b3a3
unverified
Teknium
commited on
