Commit History
Add support for GPTQ using native transformers/peft (#468)
3355706
unverified
pad_to_worst_case_seq_len boolean, for testing memory limits (#498)
8e197f6
unverified
Feat(cfg): Add code-llama configs for all sizes (#479)
3513071
unverified
Add example Llama 2 ReLoRA config (#471)
fe4d6ba
unverified
improve llama pad token handling (#475)
cb9797e
unverified
don't use mask expansion for inference (#392)
1687be6
unverified
new llama-2 default settings (#370)
fdffef5
unverified
Add wandb_entity to wandb options, update example configs, update README (#361)
7019509
unverified
set group_by_length to false in examples
36fefcf
feat/llama-2 examples (#319)
dc71d88
unverified
Add XGen info to README and example config
3881143
Use AutoTokenizer for redpajama example
945c419
Merge pull request #92 from OpenAccess-AI-Collective/flash-optimum
16bb627
unverified
Merge branch 'main' into flash-optimum
fd2c981
unverified
tweak config to work
2ba4ae8
Merge pull request #193 from OpenAccess-AI-Collective/config-fixes-20230612
94f310c
unverified
Fix config path after config moved
52cde69
config fixes
9a58e99
forgot to add this file
6b3f509
update openllama and clean up paths
d0d7eaa
more pruning
effbbf6
more config pruning and migrating
c530e4b
get rid of some configs, formalize pythioa lora config
77762a5
address PR feedback
0c6f928
linting fix
1db46a9
use pythia-12b, neox-20b is flaky
3961902
Merge pull request #132 from utensil/falcon-7b-qlora
c8242de
unverified
Trim trailing whitespace
79a8f52
unverified
Default micro_batch_size to 1 for a safer start
c9c0503
unverified
Add comments/alternatives for falcon-qlora configs
ca11ae9
unverified
falcon + qlora + xformer mbs 40 gas 2 on A6000
fb3d40f
unverified
Create config-7b-qlora.yml
72bf8aa
unverified
swap batch size for gradient accumulation steps to decouple from num gpu
c2a0792
Merge pull request #105 from viktoriussuwandi/viktoriussuwandi-patch
4df9da7
unverified
Merge pull request #106 from fearnworks/qlora-openllama-3b-example
2531ea2
unverified
Lint and format
392dfd9
Update examples/qlora-openllama-3b/README.md
6cee881
unverified
Add Readme, Clean up comments
ac85c0e
Add qlora-openllama-3b example
370d057
Update wandb_log_model on config.yml
15e57ba
unverified
Viktorius Suwandi
commited on
Update wandb_log_model on config-3b.yml
4eb68ac
unverified
Viktorius Suwandi
commited on
Update wandb_log_model on config.yml
fad06be
unverified
Viktorius Suwandi
commited on
Merge branch 'main' into refactor/rename-4b-to-gptq
147241c
unverified
sane starter defaults and add lora
157420d
add example for falcoln support
d5f944c
refactor(example): rename 4bit-lora-7b by gptq-lora-7b
7cf07fc
Thytu
commited on
refactor(param): rename load_4bit config param by gptq
dd00657
Thytu
commited on