add training details
Browse files
README.md
CHANGED
|
@@ -35,6 +35,21 @@ result = pipe(build_instruction_prompt("Perform EDA on the Iris dataset"))
|
|
| 35 |
print(result[0]['generated_text'])
|
| 36 |
```
|
| 37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
## Samples
|
| 39 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/62618f3e6dae705b2567fb13/0H8lj26xLOfLuCD0yVmER.png" width="90%"/>
|
| 40 |
|
|
|
|
| 35 |
print(result[0]['generated_text'])
|
| 36 |
```
|
| 37 |
|
| 38 |
+
## Training Details
|
| 39 |
+
lora_r: 16
|
| 40 |
+
lora_alpha: 8
|
| 41 |
+
lora_dropout: 0.05
|
| 42 |
+
target_modules: q, k, v, o, gate_proj, down_proj, up_proj, lm_head
|
| 43 |
+
weight_decay: 0
|
| 44 |
+
optmizer: paged_adamw_32bit
|
| 45 |
+
lr: 1e-4
|
| 46 |
+
lr_scheduler: cosine
|
| 47 |
+
max_seq_len: 4096
|
| 48 |
+
batch_size: 4
|
| 49 |
+
max_grad_norm: 0.5
|
| 50 |
+
warmup_ratio: 0.05
|
| 51 |
+
num_epochs: 1
|
| 52 |
+
|
| 53 |
## Samples
|
| 54 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/62618f3e6dae705b2567fb13/0H8lj26xLOfLuCD0yVmER.png" width="90%"/>
|
| 55 |
|