HamzaNaser
commited on
Commit
•
a0309bc
1
Parent(s):
236ec84
Update README.md
Browse files
README.md
CHANGED
@@ -79,7 +79,7 @@ In this section will discuss trainig procedure we followed as well metrics used
|
|
79 |
|
80 |
|
81 |
## Steps
|
82 |
-
Developing the Model ran into several steps, first step is by training the Model on 800K portion of the data twice using LoRA and traditional Full Parameters training, to see how much the performace would be affected if we used any of the PEFT techniques when training the final Model with larger split of the data, we proceed with Full param method in training as the Model accuracy were affected by the reduction of the number of trainable parameters with not much reduction in training time, might consider experimenting the use of PEFT in future project.
|
83 |
|
84 |
|
85 |
## Bleu Score Metric
|
|
|
79 |
|
80 |
|
81 |
## Steps
|
82 |
+
Developing the Model ran into several steps, first step is by training the Model on 800K portion of the data twice using LoRA and traditional Full Parameters training, to see how much the performace would be affected if we used any of the PEFT techniques when training the final Model with larger split of the data, we proceed with Full param method in training as the Model accuracy were affected by the reduction of the number of trainable parameters (from 32 to 23 Bleu score on 20K examples) with not much reduction in training time (from 7 to 6 Hrs), might consider experimenting the use of PEFT in future project.
|
83 |
|
84 |
|
85 |
## Bleu Score Metric
|