Commit History
recast loralayer, norm, lmhead + embed token weights per original qlora (#393)
96deb6b
unverified
always drop samples that are too long (#452)
50682a3
unverified
set env var for FSDP layer to wrap (#453)
5a1985b
unverified
fix eval regression caused in 13f7efaf74fcd3c4514277ccb71914c589873f6a
a213d99
is_causal fix for evals?
fbf49a4
add missing positional arg (#450)
58cf7e7
unverified
fix evals (#447)
ee26281
unverified
gracefully handle empty input (#442)
9d629d8
unverified
support user defined prompters, pretokenized datasets in config, local parquet, local arrow files (#348)
d2e7f27
unverified
disable eval using multipack for now (#437)
f733d0f
unverified
fix comma, not a tuple (#436)
008505c
unverified
use save_strategy from config if available (#434)
b3f5e00
unverified
set env for FSDP offload params (#433)
5247c50
unverified
adds color (#425)
0a22847
unverified
fix orca prompts (#422)
1b7e860
unverified
Fix(config): Update handling of deepspeed config (#404)
c01015f
unverified
fix eval steps and strategy (#403)
da10af0
unverified
better handling of empty input ids when tokenizing (#395)
85cf4f8
unverified
add utils.data.prepare_dataset
2e22404
use context manager to run things on rank0 before others (#397)
fc2d6be
unverified
don't use mask expansion for inference (#392)
1687be6
unverified
Feat(config): add max steps (#387)
3c2ad00
unverified
Added "epoch" evaluation_strategy (#388)
5d48a10
unverified
Feat(config): Add hub_strategy (#386)
73a0b6e
unverified
Error msg for sharegpt if conv has less than 2 msg (#379)
63fdb5a
unverified
don't pass rope_scaling kwarg if it's None (#383)
919246f
unverified
Fix crash when running without CUDA
15f6e57
try to detect accelerate and only use device_map=None in that case (#373)
094fc2c
unverified
fix check for flash attn branching (#377)
343ac84
unverified
remove unnecessary local variable
0c96727
simplify `load_tokenizer`
efb3b2c
improve GPU logging to break out pytorch cache and system mem
7b55fe6
quiet noise from llama tokenizer by setting pad token earlier
e029ab3
extract module for working with cfg
8cec513
fix DefaultDict.__or__
a13e45d
Attention mask and position id fixes for packing (#285)
2bb0b78
unverified
Add wandb_entity to wandb options, update example configs, update README (#361)
7019509
unverified
Fix(model loading): Warn when model revision is passed to gptq (#364)
96bd6ae
unverified
Fix(message): Improve error message for bad format (#365)
e37d935
unverified
Feat: Add rope scaling (#343)
b521206
unverified
Merge pull request #356 from tmm1/load_model-args
11ddccb
unverified
simplify load_model signature
7181022
log GPU memory usage
e303d64
ensure enable_input_require_grads is called on model before getting the peft model (#345)
176b888
unverified
experimental llama 2 chat support (#296)
3392270
unverified
Jan Philipp Harries
Jan Philipp Harries
commited on
Update XFormers Attention Monkeypatch to handle Llama-2 70B (GQA) (#339)
10405b9
unverified
ssmi153
commited on
Added Orca Mini prompt strategy (#263)
c93655c
unverified
Jan Philipp Harries
Jan Philipp Harries
commited on