Hugging Face's logo

caldana
/
hc-mistral-alpaca

PEFT
Safetensors
mistral
axolotl
Generated from Trainer
4-bit precision
bitsandbytes
Model card Files Community
caldana commited on
Commit
81396c4
·
verified ·
1 Parent(s): 130d957

End of training

Browse files
Files changed (1) hide show
  1. README.md +105 -15
README.md CHANGED
@@ -67,7 +67,7 @@ wandb_entity:
67
  gradient_accumulation_steps: 4
68
  micro_batch_size: 16
69
  eval_batch_size: 16
70
- num_epochs: 10
71
  optimizer: adamw_bnb_8bit
72
  lr_scheduler: cosine
73
  learning_rate: 0.0002
@@ -116,7 +116,7 @@ save_safetensors: true
116
 
117
  This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
118
  It achieves the following results on the evaluation set:
119
- - Loss: 0.3253
120
 
121
  ## Model description
122
 
@@ -144,22 +144,112 @@ The following hyperparameters were used during training:
144
  - optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-05
145
  - lr_scheduler_type: cosine
146
  - lr_scheduler_warmup_steps: 20
147
- - num_epochs: 10
148
 
149
  ### Training results
150
 
151
- | Training Loss | Epoch | Step | Validation Loss |
152
- |:-------------:|:------:|:----:|:---------------:|
153
- | 1.334 | 0.6667 | 1 | 1.2849 |
154
- | 1.3476 | 1.3333 | 2 | 1.2762 |
155
- | 1.2977 | 2.0 | 3 | 1.2492 |
156
- | 1.3157 | 2.6667 | 4 | 1.1859 |
157
- | 1.1755 | 3.3333 | 5 | 1.0709 |
158
- | 1.1377 | 4.0 | 6 | 0.9092 |
159
- | 0.9404 | 4.6667 | 7 | 0.7201 |
160
- | 0.7404 | 5.3333 | 8 | 0.5605 |
161
- | 0.5547 | 6.0 | 9 | 0.4305 |
162
- | 0.4057 | 6.6667 | 10 | 0.3253 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
163
 
164
 
165
  ### Framework versions
 
67
  gradient_accumulation_steps: 4
68
  micro_batch_size: 16
69
  eval_batch_size: 16
70
+ num_epochs: 100
71
  optimizer: adamw_bnb_8bit
72
  lr_scheduler: cosine
73
  learning_rate: 0.0002
 
116
 
117
  This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
118
  It achieves the following results on the evaluation set:
119
+ - Loss: 0.3648
120
 
121
  ## Model description
122
 
 
144
  - optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-05
145
  - lr_scheduler_type: cosine
146
  - lr_scheduler_warmup_steps: 20
147
+ - num_epochs: 100
148
 
149
  ### Training results
150
 
151
+ | Training Loss | Epoch | Step | Validation Loss |
152
+ |:-------------:|:-------:|:----:|:---------------:|
153
+ | 1.334 | 0.6667 | 1 | 1.2849 |
154
+ | 1.3476 | 1.3333 | 2 | 1.2780 |
155
+ | 1.2981 | 2.0 | 3 | 1.2487 |
156
+ | 1.3157 | 2.6667 | 4 | 1.1840 |
157
+ | 1.1757 | 3.3333 | 5 | 1.0690 |
158
+ | 1.1376 | 4.0 | 6 | 0.9086 |
159
+ | 0.9395 | 4.6667 | 7 | 0.7184 |
160
+ | 0.7385 | 5.3333 | 8 | 0.5617 |
161
+ | 0.5541 | 6.0 | 9 | 0.4307 |
162
+ | 0.4056 | 6.6667 | 10 | 0.3257 |
163
+ | 0.2791 | 7.3333 | 11 | 0.2866 |
164
+ | 0.2198 | 8.0 | 12 | 0.2453 |
165
+ | 0.1746 | 8.6667 | 13 | 0.2167 |
166
+ | 0.1582 | 9.3333 | 14 | 0.2104 |
167
+ | 0.1515 | 10.0 | 15 | 0.1699 |
168
+ | 0.1168 | 10.6667 | 16 | 0.1502 |
169
+ | 0.087 | 11.3333 | 17 | 0.1415 |
170
+ | 0.1 | 12.0 | 18 | 0.1574 |
171
+ | 0.0832 | 12.6667 | 19 | 0.1699 |
172
+ | 0.0765 | 13.3333 | 20 | 0.1601 |
173
+ | 0.0697 | 14.0 | 21 | 0.1544 |
174
+ | 0.0625 | 14.6667 | 22 | 0.1653 |
175
+ | 0.0583 | 15.3333 | 23 | 0.1628 |
176
+ | 0.047 | 16.0 | 24 | 0.1463 |
177
+ | 0.0366 | 16.6667 | 25 | 0.1637 |
178
+ | 0.0342 | 17.3333 | 26 | 0.2020 |
179
+ | 0.0398 | 18.0 | 27 | 0.1801 |
180
+ | 0.0319 | 18.6667 | 28 | 0.1835 |
181
+ | 0.0229 | 19.3333 | 29 | 0.1957 |
182
+ | 0.0286 | 20.0 | 30 | 0.2024 |
183
+ | 0.0166 | 20.6667 | 31 | 0.2519 |
184
+ | 0.0184 | 21.3333 | 32 | 0.2699 |
185
+ | 0.0129 | 22.0 | 33 | 0.2813 |
186
+ | 0.0109 | 22.6667 | 34 | 0.2950 |
187
+ | 0.0105 | 23.3333 | 35 | 0.3037 |
188
+ | 0.0111 | 24.0 | 36 | 0.3161 |
189
+ | 0.0071 | 24.6667 | 37 | 0.3310 |
190
+ | 0.0115 | 25.3333 | 38 | 0.3375 |
191
+ | 0.0051 | 26.0 | 39 | 0.3456 |
192
+ | 0.004 | 26.6667 | 40 | 0.3488 |
193
+ | 0.0077 | 27.3333 | 41 | 0.3599 |
194
+ | 0.0028 | 28.0 | 42 | 0.3706 |
195
+ | 0.0021 | 28.6667 | 43 | 0.3737 |
196
+ | 0.002 | 29.3333 | 44 | 0.3729 |
197
+ | 0.0017 | 30.0 | 45 | 0.3742 |
198
+ | 0.0013 | 30.6667 | 46 | 0.3757 |
199
+ | 0.0004 | 31.3333 | 47 | 0.3755 |
200
+ | 0.0006 | 32.0 | 48 | 0.3764 |
201
+ | 0.0002 | 32.6667 | 49 | 0.3750 |
202
+ | 0.0011 | 33.3333 | 50 | 0.3646 |
203
+ | 0.0005 | 34.0 | 51 | 0.3586 |
204
+ | 0.0013 | 34.6667 | 52 | 0.3617 |
205
+ | 0.0005 | 35.3333 | 53 | 0.3638 |
206
+ | 0.0011 | 36.0 | 54 | 0.3657 |
207
+ | 0.0003 | 36.6667 | 55 | 0.3710 |
208
+ | 0.0002 | 37.3333 | 56 | 0.3711 |
209
+ | 0.0004 | 38.0 | 57 | 0.3736 |
210
+ | 0.0003 | 38.6667 | 58 | 0.3784 |
211
+ | 0.0001 | 39.3333 | 59 | 0.3795 |
212
+ | 0.0007 | 40.0 | 60 | 0.3737 |
213
+ | 0.0001 | 40.6667 | 61 | 0.3730 |
214
+ | 0.0003 | 41.3333 | 62 | 0.3729 |
215
+ | 0.0002 | 42.0 | 63 | 0.3714 |
216
+ | 0.0001 | 42.6667 | 64 | 0.3698 |
217
+ | 0.0001 | 43.3333 | 65 | 0.3704 |
218
+ | 0.0001 | 44.0 | 66 | 0.3704 |
219
+ | 0.0001 | 44.6667 | 67 | 0.3705 |
220
+ | 0.0001 | 45.3333 | 68 | 0.3655 |
221
+ | 0.0002 | 46.0 | 69 | 0.3672 |
222
+ | 0.0002 | 46.6667 | 70 | 0.3682 |
223
+ | 0.0002 | 47.3333 | 71 | 0.3656 |
224
+ | 0.0001 | 48.0 | 72 | 0.3663 |
225
+ | 0.0001 | 48.6667 | 73 | 0.3668 |
226
+ | 0.0001 | 49.3333 | 74 | 0.3673 |
227
+ | 0.0001 | 50.0 | 75 | 0.3638 |
228
+ | 0.0001 | 50.6667 | 76 | 0.3640 |
229
+ | 0.0001 | 51.3333 | 77 | 0.3643 |
230
+ | 0.0001 | 52.0 | 78 | 0.3640 |
231
+ | 0.0001 | 52.6667 | 79 | 0.3648 |
232
+ | 0.0001 | 53.3333 | 80 | 0.3629 |
233
+ | 0.0001 | 54.0 | 81 | 0.3648 |
234
+ | 0.0001 | 54.6667 | 82 | 0.3617 |
235
+ | 0.0001 | 55.3333 | 83 | 0.3632 |
236
+ | 0.0001 | 56.0 | 84 | 0.3650 |
237
+ | 0.0001 | 56.6667 | 85 | 0.3636 |
238
+ | 0.0001 | 57.3333 | 86 | 0.3633 |
239
+ | 0.0001 | 58.0 | 87 | 0.3673 |
240
+ | 0.0001 | 58.6667 | 88 | 0.3663 |
241
+ | 0.0001 | 59.3333 | 89 | 0.3618 |
242
+ | 0.0001 | 60.0 | 90 | 0.3635 |
243
+ | 0.0001 | 60.6667 | 91 | 0.3605 |
244
+ | 0.0001 | 61.3333 | 92 | 0.3654 |
245
+ | 0.0001 | 62.0 | 93 | 0.3647 |
246
+ | 0.0001 | 62.6667 | 94 | 0.3586 |
247
+ | 0.0001 | 63.3333 | 95 | 0.3601 |
248
+ | 0.0001 | 64.0 | 96 | 0.3631 |
249
+ | 0.0001 | 64.6667 | 97 | 0.3629 |
250
+ | 0.0001 | 65.3333 | 98 | 0.3652 |
251
+ | 0.0001 | 66.0 | 99 | 0.3645 |
252
+ | 0.0001 | 66.6667 | 100 | 0.3648 |
253
 
254
 
255
  ### Framework versions