Commit History
don't pass rope_scaling kwarg if it's None (#383)
919246f
unverified
Fix crash when running without CUDA
15f6e57
try to detect accelerate and only use device_map=None in that case (#373)
094fc2c
unverified
fix check for flash attn branching (#377)
343ac84
unverified
remove unnecessary local variable
0c96727
simplify `load_tokenizer`
efb3b2c
improve GPU logging to break out pytorch cache and system mem
7b55fe6
quiet noise from llama tokenizer by setting pad token earlier
e029ab3
extract module for working with cfg
8cec513
fix DefaultDict.__or__
a13e45d
Attention mask and position id fixes for packing (#285)
2bb0b78
unverified
Add wandb_entity to wandb options, update example configs, update README (#361)
7019509
unverified
Fix(model loading): Warn when model revision is passed to gptq (#364)
96bd6ae
unverified
Fix(message): Improve error message for bad format (#365)
e37d935
unverified
Feat: Add rope scaling (#343)
b521206
unverified
Merge pull request #356 from tmm1/load_model-args
11ddccb
unverified
simplify load_model signature
7181022
log GPU memory usage
e303d64
ensure enable_input_require_grads is called on model before getting the peft model (#345)
176b888
unverified
experimental llama 2 chat support (#296)
3392270
unverified
Jan Philipp Harries
Jan Philipp Harries
commited on
Update XFormers Attention Monkeypatch to handle Llama-2 70B (GQA) (#339)
10405b9
unverified
ssmi153
commited on
Added Orca Mini prompt strategy (#263)
c93655c
unverified
Jan Philipp Harries
Jan Philipp Harries
commited on
optimize the iteration when tokenizeing large datasets (#332)
fe28543
unverified
fix typo
2eda9e0
scope flash-attn+qlora fix correctly, scope to llama, add comment
78b9efb
move flash-attn monkey patch alongside the others
312a9fa
ensure flash-attn fixes happen in both adapter/lora modes, and use torch_dtype
248bf90
qlora w flash attention fixes (#333)
77085ea
unverified
add peft install back since it doesn't get installed by setup.py (#331)
db2a358
unverified
update prompts for open orca to match the paper (#317)
3d4984b
unverified
Merge pull request #307 from OpenAccess-AI-Collective/xgen-user-sharegpt-tokens
40a53ff
unverified
Merge pull request #313 from OpenAccess-AI-Collective/tokenizer-llama2-embeddings
3ffb018
unverified
don't resize embeddings to multiples of 32x by default
1066751
better handling since xgen tokenizer breaks with convert_tokens_to_ids
2a428e8
flash attention 2
9b790d3
fix sdp attention to use the flash/mem-efficient context manaager
a032c9f
feat: use multi-core
45ac7c4
fix axolotl training args dataclass annotation
ebaec3c
misc fixes
d75adb9
Merge pull request #276 from theobjectivedad/logging_enhancement
6f16c45
unverified
Fixed pre-commit problems, fixed small bug in logging_config to handle LOG_LEVEL env var
b1f4f7a
Merge branch 'OpenAccess-AI-Collective:main' into logging_enhancement
83237b8
unverified
The Objective Dad
commited on