Commit History
more sane defaults for openllama 3b used for quickstarts (#602)
674c576
unverified
btlm and falcon monkey patches for flash attn (#566)
6b9b229
unverified
make phi training work with Loras (#588)
62eaee7
unverified
Support Sample packing for phi arch (#586)
12a2dbb
unverified
Fix Codellama examples (#582)
1aa4007
unverified
Doan Minh Phuong
commited on
Phi examples (#569)
2284209
unverified
Add training callback to send predictions to WandB table (#521)
5b67ea9
unverified
recommend padding when using sample packing (#531)
3437149
unverified
Add support for GPTQ using native transformers/peft (#468)
3355706
unverified
pad_to_worst_case_seq_len boolean, for testing memory limits (#498)
8e197f6
unverified
Feat(cfg): Add code-llama configs for all sizes (#479)
3513071
unverified
Add example Llama 2 ReLoRA config (#471)
fe4d6ba
unverified
improve llama pad token handling (#475)
cb9797e
unverified
don't use mask expansion for inference (#392)
1687be6
unverified
new llama-2 default settings (#370)
fdffef5
unverified
Add wandb_entity to wandb options, update example configs, update README (#361)
7019509
unverified
set group_by_length to false in examples
36fefcf
feat/llama-2 examples (#319)
dc71d88
unverified
Add XGen info to README and example config
3881143
Use AutoTokenizer for redpajama example
945c419
Merge pull request #92 from OpenAccess-AI-Collective/flash-optimum
16bb627
unverified
Merge branch 'main' into flash-optimum
fd2c981
unverified
tweak config to work
2ba4ae8
Merge pull request #193 from OpenAccess-AI-Collective/config-fixes-20230612
94f310c
unverified
Fix config path after config moved
52cde69
config fixes
9a58e99
forgot to add this file
6b3f509
update openllama and clean up paths
d0d7eaa
more pruning
effbbf6
more config pruning and migrating
c530e4b
get rid of some configs, formalize pythioa lora config
77762a5
address PR feedback
0c6f928
linting fix
1db46a9
use pythia-12b, neox-20b is flaky
3961902
Merge pull request #132 from utensil/falcon-7b-qlora
c8242de
unverified
Trim trailing whitespace
79a8f52
unverified
Default micro_batch_size to 1 for a safer start
c9c0503
unverified
Add comments/alternatives for falcon-qlora configs
ca11ae9
unverified
falcon + qlora + xformer mbs 40 gas 2 on A6000
fb3d40f
unverified
Create config-7b-qlora.yml
72bf8aa
unverified
swap batch size for gradient accumulation steps to decouple from num gpu
c2a0792
Merge pull request #105 from viktoriussuwandi/viktoriussuwandi-patch
4df9da7
unverified
Merge pull request #106 from fearnworks/qlora-openllama-3b-example
2531ea2
unverified
Lint and format
392dfd9
Update examples/qlora-openllama-3b/README.md
6cee881
unverified
Add Readme, Clean up comments
ac85c0e
Add qlora-openllama-3b example
370d057
Update wandb_log_model on config.yml
15e57ba
unverified
Viktorius Suwandi
commited on