| library_name: peft | |
| tags: | |
| - peft | |
| - product | |
| - LoRA | |
| base_model: bigscience/bloom-3b | |
| ## Training procedure | |
| per_device_train_batch_size=4, | |
| gradient_accumulation_steps=4, | |
| warmup_steps=100, | |
| max_steps=100, | |
| learning_rate=1e-3, | |
| fp16=True, | |
| logging_steps=1, | |
| output_dir='outputs' | |
| ### Framework versions | |
| - PEFT 0.5.0.dev0 |