Shaier commited on
Commit
1088975
·
1 Parent(s): 1ba0eba

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +222 -52
README.md CHANGED
@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
13
 
14
  This model is a fine-tuned version of [Shaier/distilbert-base-uncased-continued_training-medqa](https://huggingface.co/Shaier/distilbert-base-uncased-continued_training-medqa) on an unknown dataset.
15
  It achieves the following results on the evaluation set:
16
- - Loss: 0.4063
17
 
18
  ## Model description
19
 
@@ -41,63 +41,233 @@ The following hyperparameters were used during training:
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
  - lr_scheduler_warmup_steps: 100
44
- - num_epochs: 50
45
  - mixed_precision_training: Native AMP
46
 
47
  ### Training results
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:-----:|:---------------:|
51
- | No log | 1.0 | 333 | 0.4659 |
52
- | No log | 2.0 | 666 | 0.4547 |
53
- | No log | 3.0 | 999 | 0.3882 |
54
- | No log | 4.0 | 1332 | 0.4310 |
55
- | No log | 5.0 | 1665 | 0.4194 |
56
- | No log | 6.0 | 1998 | 0.5209 |
57
- | No log | 7.0 | 2331 | 0.4812 |
58
- | 0.4829 | 8.0 | 2664 | 0.5321 |
59
- | 0.4829 | 9.0 | 2997 | 0.3646 |
60
- | 0.4829 | 10.0 | 3330 | 0.4339 |
61
- | 0.4829 | 11.0 | 3663 | 0.5188 |
62
- | 0.4829 | 12.0 | 3996 | 0.4148 |
63
- | 0.4829 | 13.0 | 4329 | 0.4615 |
64
- | 0.4829 | 14.0 | 4662 | 0.3825 |
65
- | 0.4829 | 15.0 | 4995 | 0.4617 |
66
- | 0.4773 | 16.0 | 5328 | 0.3400 |
67
- | 0.4773 | 17.0 | 5661 | 0.4740 |
68
- | 0.4773 | 18.0 | 5994 | 0.5057 |
69
- | 0.4773 | 19.0 | 6327 | 0.5477 |
70
- | 0.4773 | 20.0 | 6660 | 0.4426 |
71
- | 0.4773 | 21.0 | 6993 | 0.3574 |
72
- | 0.4773 | 22.0 | 7326 | 0.4031 |
73
- | 0.4773 | 23.0 | 7659 | 0.4491 |
74
- | 0.4715 | 24.0 | 7992 | 0.4340 |
75
- | 0.4715 | 25.0 | 8325 | 0.4602 |
76
- | 0.4715 | 26.0 | 8658 | 0.4659 |
77
- | 0.4715 | 27.0 | 8991 | 0.4321 |
78
- | 0.4715 | 28.0 | 9324 | 0.4335 |
79
- | 0.4715 | 29.0 | 9657 | 0.4458 |
80
- | 0.4715 | 30.0 | 9990 | 0.4285 |
81
- | 0.4715 | 31.0 | 10323 | 0.5002 |
82
- | 0.4671 | 32.0 | 10656 | 0.4706 |
83
- | 0.4671 | 33.0 | 10989 | 0.5368 |
84
- | 0.4671 | 34.0 | 11322 | 0.4028 |
85
- | 0.4671 | 35.0 | 11655 | 0.5171 |
86
- | 0.4671 | 36.0 | 11988 | 0.4506 |
87
- | 0.4671 | 37.0 | 12321 | 0.4163 |
88
- | 0.4671 | 38.0 | 12654 | 0.4905 |
89
- | 0.4671 | 39.0 | 12987 | 0.5168 |
90
- | 0.4646 | 40.0 | 13320 | 0.4412 |
91
- | 0.4646 | 41.0 | 13653 | 0.4773 |
92
- | 0.4646 | 42.0 | 13986 | 0.4835 |
93
- | 0.4646 | 43.0 | 14319 | 0.4716 |
94
- | 0.4646 | 44.0 | 14652 | 0.4431 |
95
- | 0.4646 | 45.0 | 14985 | 0.4187 |
96
- | 0.4646 | 46.0 | 15318 | 0.3389 |
97
- | 0.4646 | 47.0 | 15651 | 0.4699 |
98
- | 0.4628 | 48.0 | 15984 | 0.4880 |
99
- | 0.4628 | 49.0 | 16317 | 0.5058 |
100
- | 0.4628 | 50.0 | 16650 | 0.4275 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
101
 
102
 
103
  ### Framework versions
 
13
 
14
  This model is a fine-tuned version of [Shaier/distilbert-base-uncased-continued_training-medqa](https://huggingface.co/Shaier/distilbert-base-uncased-continued_training-medqa) on an unknown dataset.
15
  It achieves the following results on the evaluation set:
16
+ - Loss: 0.5389
17
 
18
  ## Model description
19
 
 
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
  - lr_scheduler_warmup_steps: 100
44
+ - num_epochs: 220
45
  - mixed_precision_training: Native AMP
46
 
47
  ### Training results
48
 
49
  | Training Loss | Epoch | Step | Validation Loss |
50
  |:-------------:|:-----:|:-----:|:---------------:|
51
+ | No log | 1.0 | 333 | 0.4516 |
52
+ | No log | 2.0 | 666 | 0.4277 |
53
+ | No log | 3.0 | 999 | 0.3734 |
54
+ | No log | 4.0 | 1332 | 0.4083 |
55
+ | No log | 5.0 | 1665 | 0.4134 |
56
+ | No log | 6.0 | 1998 | 0.5093 |
57
+ | No log | 7.0 | 2331 | 0.4639 |
58
+ | 0.4564 | 8.0 | 2664 | 0.5132 |
59
+ | 0.4564 | 9.0 | 2997 | 0.3483 |
60
+ | 0.4564 | 10.0 | 3330 | 0.4174 |
61
+ | 0.4564 | 11.0 | 3663 | 0.4975 |
62
+ | 0.4564 | 12.0 | 3996 | 0.4030 |
63
+ | 0.4564 | 13.0 | 4329 | 0.4476 |
64
+ | 0.4564 | 14.0 | 4662 | 0.3692 |
65
+ | 0.4564 | 15.0 | 4995 | 0.4474 |
66
+ | 0.4533 | 16.0 | 5328 | 0.3289 |
67
+ | 0.4533 | 17.0 | 5661 | 0.4647 |
68
+ | 0.4533 | 18.0 | 5994 | 0.4873 |
69
+ | 0.4533 | 19.0 | 6327 | 0.5323 |
70
+ | 0.4533 | 20.0 | 6660 | 0.4273 |
71
+ | 0.4533 | 21.0 | 6993 | 0.3426 |
72
+ | 0.4533 | 22.0 | 7326 | 0.3892 |
73
+ | 0.4533 | 23.0 | 7659 | 0.4297 |
74
+ | 0.4493 | 24.0 | 7992 | 0.4162 |
75
+ | 0.4493 | 25.0 | 8325 | 0.4424 |
76
+ | 0.4493 | 26.0 | 8658 | 0.4575 |
77
+ | 0.4493 | 27.0 | 8991 | 0.4192 |
78
+ | 0.4493 | 28.0 | 9324 | 0.4151 |
79
+ | 0.4493 | 29.0 | 9657 | 0.4321 |
80
+ | 0.4493 | 30.0 | 9990 | 0.4129 |
81
+ | 0.4493 | 31.0 | 10323 | 0.4869 |
82
+ | 0.4456 | 32.0 | 10656 | 0.4510 |
83
+ | 0.4456 | 33.0 | 10989 | 0.5263 |
84
+ | 0.4456 | 34.0 | 11322 | 0.3908 |
85
+ | 0.4456 | 35.0 | 11655 | 0.5016 |
86
+ | 0.4456 | 36.0 | 11988 | 0.4454 |
87
+ | 0.4456 | 37.0 | 12321 | 0.4011 |
88
+ | 0.4456 | 38.0 | 12654 | 0.4714 |
89
+ | 0.4456 | 39.0 | 12987 | 0.4972 |
90
+ | 0.443 | 40.0 | 13320 | 0.4200 |
91
+ | 0.443 | 41.0 | 13653 | 0.4659 |
92
+ | 0.443 | 42.0 | 13986 | 0.4758 |
93
+ | 0.443 | 43.0 | 14319 | 0.4509 |
94
+ | 0.443 | 44.0 | 14652 | 0.4211 |
95
+ | 0.443 | 45.0 | 14985 | 0.4007 |
96
+ | 0.443 | 46.0 | 15318 | 0.3205 |
97
+ | 0.443 | 47.0 | 15651 | 0.4479 |
98
+ | 0.4402 | 48.0 | 15984 | 0.4723 |
99
+ | 0.4402 | 49.0 | 16317 | 0.4956 |
100
+ | 0.4402 | 50.0 | 16650 | 0.4103 |
101
+ | 0.4402 | 51.0 | 16983 | 0.4234 |
102
+ | 0.4402 | 52.0 | 17316 | 0.4052 |
103
+ | 0.4402 | 53.0 | 17649 | 0.4033 |
104
+ | 0.4402 | 54.0 | 17982 | 0.4139 |
105
+ | 0.4402 | 55.0 | 18315 | 0.3618 |
106
+ | 0.4372 | 56.0 | 18648 | 0.5102 |
107
+ | 0.4372 | 57.0 | 18981 | 0.4166 |
108
+ | 0.4372 | 58.0 | 19314 | 0.4475 |
109
+ | 0.4372 | 59.0 | 19647 | 0.4259 |
110
+ | 0.4372 | 60.0 | 19980 | 0.4018 |
111
+ | 0.4372 | 61.0 | 20313 | 0.5005 |
112
+ | 0.4372 | 62.0 | 20646 | 0.4445 |
113
+ | 0.4372 | 63.0 | 20979 | 0.4280 |
114
+ | 0.434 | 64.0 | 21312 | 0.4533 |
115
+ | 0.434 | 65.0 | 21645 | 0.3672 |
116
+ | 0.434 | 66.0 | 21978 | 0.4726 |
117
+ | 0.434 | 67.0 | 22311 | 0.4084 |
118
+ | 0.434 | 68.0 | 22644 | 0.4508 |
119
+ | 0.434 | 69.0 | 22977 | 0.3746 |
120
+ | 0.434 | 70.0 | 23310 | 0.4703 |
121
+ | 0.434 | 71.0 | 23643 | 0.4789 |
122
+ | 0.4314 | 72.0 | 23976 | 0.3963 |
123
+ | 0.4314 | 73.0 | 24309 | 0.3800 |
124
+ | 0.4314 | 74.0 | 24642 | 0.5051 |
125
+ | 0.4314 | 75.0 | 24975 | 0.4245 |
126
+ | 0.4314 | 76.0 | 25308 | 0.4745 |
127
+ | 0.4314 | 77.0 | 25641 | 0.4351 |
128
+ | 0.4314 | 78.0 | 25974 | 0.4367 |
129
+ | 0.4314 | 79.0 | 26307 | 0.4200 |
130
+ | 0.4291 | 80.0 | 26640 | 0.4985 |
131
+ | 0.4291 | 81.0 | 26973 | 0.5058 |
132
+ | 0.4291 | 82.0 | 27306 | 0.4154 |
133
+ | 0.4291 | 83.0 | 27639 | 0.4837 |
134
+ | 0.4291 | 84.0 | 27972 | 0.3865 |
135
+ | 0.4291 | 85.0 | 28305 | 0.4357 |
136
+ | 0.4291 | 86.0 | 28638 | 0.3978 |
137
+ | 0.4291 | 87.0 | 28971 | 0.4413 |
138
+ | 0.4263 | 88.0 | 29304 | 0.4223 |
139
+ | 0.4263 | 89.0 | 29637 | 0.4241 |
140
+ | 0.4263 | 90.0 | 29970 | 0.4525 |
141
+ | 0.4263 | 91.0 | 30303 | 0.3895 |
142
+ | 0.4263 | 92.0 | 30636 | 0.4207 |
143
+ | 0.4263 | 93.0 | 30969 | 0.3217 |
144
+ | 0.4263 | 94.0 | 31302 | 0.3725 |
145
+ | 0.4263 | 95.0 | 31635 | 0.4354 |
146
+ | 0.4239 | 96.0 | 31968 | 0.4169 |
147
+ | 0.4239 | 97.0 | 32301 | 0.4873 |
148
+ | 0.4239 | 98.0 | 32634 | 0.4219 |
149
+ | 0.4239 | 99.0 | 32967 | 0.4984 |
150
+ | 0.4239 | 100.0 | 33300 | 0.4078 |
151
+ | 0.4239 | 101.0 | 33633 | 0.4463 |
152
+ | 0.4239 | 102.0 | 33966 | 0.3371 |
153
+ | 0.4239 | 103.0 | 34299 | 0.3896 |
154
+ | 0.422 | 104.0 | 34632 | 0.4743 |
155
+ | 0.422 | 105.0 | 34965 | 0.4931 |
156
+ | 0.422 | 106.0 | 35298 | 0.3574 |
157
+ | 0.422 | 107.0 | 35631 | 0.4127 |
158
+ | 0.422 | 108.0 | 35964 | 0.3892 |
159
+ | 0.422 | 109.0 | 36297 | 0.3881 |
160
+ | 0.422 | 110.0 | 36630 | 0.4221 |
161
+ | 0.422 | 111.0 | 36963 | 0.3924 |
162
+ | 0.4204 | 112.0 | 37296 | 0.4067 |
163
+ | 0.4204 | 113.0 | 37629 | 0.4357 |
164
+ | 0.4204 | 114.0 | 37962 | 0.4175 |
165
+ | 0.4204 | 115.0 | 38295 | 0.4424 |
166
+ | 0.4204 | 116.0 | 38628 | 0.3925 |
167
+ | 0.4204 | 117.0 | 38961 | 0.4693 |
168
+ | 0.4204 | 118.0 | 39294 | 0.3503 |
169
+ | 0.4204 | 119.0 | 39627 | 0.4761 |
170
+ | 0.4183 | 120.0 | 39960 | 0.3816 |
171
+ | 0.4183 | 121.0 | 40293 | 0.3903 |
172
+ | 0.4183 | 122.0 | 40626 | 0.3535 |
173
+ | 0.4183 | 123.0 | 40959 | 0.4388 |
174
+ | 0.4183 | 124.0 | 41292 | 0.4519 |
175
+ | 0.4183 | 125.0 | 41625 | 0.4241 |
176
+ | 0.4183 | 126.0 | 41958 | 0.4085 |
177
+ | 0.4183 | 127.0 | 42291 | 0.4836 |
178
+ | 0.4168 | 128.0 | 42624 | 0.4101 |
179
+ | 0.4168 | 129.0 | 42957 | 0.4749 |
180
+ | 0.4168 | 130.0 | 43290 | 0.4022 |
181
+ | 0.4168 | 131.0 | 43623 | 0.4861 |
182
+ | 0.4168 | 132.0 | 43956 | 0.4376 |
183
+ | 0.4168 | 133.0 | 44289 | 0.4597 |
184
+ | 0.4168 | 134.0 | 44622 | 0.4154 |
185
+ | 0.4168 | 135.0 | 44955 | 0.4431 |
186
+ | 0.415 | 136.0 | 45288 | 0.4887 |
187
+ | 0.415 | 137.0 | 45621 | 0.4229 |
188
+ | 0.415 | 138.0 | 45954 | 0.3997 |
189
+ | 0.415 | 139.0 | 46287 | 0.4185 |
190
+ | 0.415 | 140.0 | 46620 | 0.4633 |
191
+ | 0.415 | 141.0 | 46953 | 0.4061 |
192
+ | 0.415 | 142.0 | 47286 | 0.4604 |
193
+ | 0.415 | 143.0 | 47619 | 0.4047 |
194
+ | 0.4139 | 144.0 | 47952 | 0.4272 |
195
+ | 0.4139 | 145.0 | 48285 | 0.4783 |
196
+ | 0.4139 | 146.0 | 48618 | 0.3954 |
197
+ | 0.4139 | 147.0 | 48951 | 0.4501 |
198
+ | 0.4139 | 148.0 | 49284 | 0.4941 |
199
+ | 0.4139 | 149.0 | 49617 | 0.4112 |
200
+ | 0.4139 | 150.0 | 49950 | 0.4582 |
201
+ | 0.4139 | 151.0 | 50283 | 0.4361 |
202
+ | 0.4126 | 152.0 | 50616 | 0.3535 |
203
+ | 0.4126 | 153.0 | 50949 | 0.3797 |
204
+ | 0.4126 | 154.0 | 51282 | 0.4080 |
205
+ | 0.4126 | 155.0 | 51615 | 0.4049 |
206
+ | 0.4126 | 156.0 | 51948 | 0.4255 |
207
+ | 0.4126 | 157.0 | 52281 | 0.4303 |
208
+ | 0.4126 | 158.0 | 52614 | 0.4950 |
209
+ | 0.4126 | 159.0 | 52947 | 0.3721 |
210
+ | 0.4114 | 160.0 | 53280 | 0.2861 |
211
+ | 0.4114 | 161.0 | 53613 | 0.3775 |
212
+ | 0.4114 | 162.0 | 53946 | 0.4274 |
213
+ | 0.4114 | 163.0 | 54279 | 0.3904 |
214
+ | 0.4114 | 164.0 | 54612 | 0.4687 |
215
+ | 0.4114 | 165.0 | 54945 | 0.4013 |
216
+ | 0.4114 | 166.0 | 55278 | 0.4760 |
217
+ | 0.4114 | 167.0 | 55611 | 0.3554 |
218
+ | 0.4104 | 168.0 | 55944 | 0.5193 |
219
+ | 0.4104 | 169.0 | 56277 | 0.4476 |
220
+ | 0.4104 | 170.0 | 56610 | 0.5011 |
221
+ | 0.4104 | 171.0 | 56943 | 0.4441 |
222
+ | 0.4104 | 172.0 | 57276 | 0.4457 |
223
+ | 0.4104 | 173.0 | 57609 | 0.3792 |
224
+ | 0.4104 | 174.0 | 57942 | 0.5116 |
225
+ | 0.4104 | 175.0 | 58275 | 0.4249 |
226
+ | 0.4097 | 176.0 | 58608 | 0.3804 |
227
+ | 0.4097 | 177.0 | 58941 | 0.3886 |
228
+ | 0.4097 | 178.0 | 59274 | 0.4420 |
229
+ | 0.4097 | 179.0 | 59607 | 0.3573 |
230
+ | 0.4097 | 180.0 | 59940 | 0.3635 |
231
+ | 0.4097 | 181.0 | 60273 | 0.4596 |
232
+ | 0.4097 | 182.0 | 60606 | 0.3674 |
233
+ | 0.4097 | 183.0 | 60939 | 0.3869 |
234
+ | 0.409 | 184.0 | 61272 | 0.3909 |
235
+ | 0.409 | 185.0 | 61605 | 0.4339 |
236
+ | 0.409 | 186.0 | 61938 | 0.4475 |
237
+ | 0.409 | 187.0 | 62271 | 0.3218 |
238
+ | 0.409 | 188.0 | 62604 | 0.3771 |
239
+ | 0.409 | 189.0 | 62937 | 0.4007 |
240
+ | 0.409 | 190.0 | 63270 | 0.4520 |
241
+ | 0.409 | 191.0 | 63603 | 0.3980 |
242
+ | 0.4077 | 192.0 | 63936 | 0.4572 |
243
+ | 0.4077 | 193.0 | 64269 | 0.3952 |
244
+ | 0.4077 | 194.0 | 64602 | 0.4384 |
245
+ | 0.4077 | 195.0 | 64935 | 0.4795 |
246
+ | 0.4077 | 196.0 | 65268 | 0.3743 |
247
+ | 0.4077 | 197.0 | 65601 | 0.4445 |
248
+ | 0.4077 | 198.0 | 65934 | 0.3925 |
249
+ | 0.4077 | 199.0 | 66267 | 0.4564 |
250
+ | 0.4075 | 200.0 | 66600 | 0.4580 |
251
+ | 0.4075 | 201.0 | 66933 | 0.4446 |
252
+ | 0.4075 | 202.0 | 67266 | 0.4289 |
253
+ | 0.4075 | 203.0 | 67599 | 0.3722 |
254
+ | 0.4075 | 204.0 | 67932 | 0.4810 |
255
+ | 0.4075 | 205.0 | 68265 | 0.4004 |
256
+ | 0.4075 | 206.0 | 68598 | 0.4219 |
257
+ | 0.4075 | 207.0 | 68931 | 0.3926 |
258
+ | 0.407 | 208.0 | 69264 | 0.6043 |
259
+ | 0.407 | 209.0 | 69597 | 0.3835 |
260
+ | 0.407 | 210.0 | 69930 | 0.3791 |
261
+ | 0.407 | 211.0 | 70263 | 0.4152 |
262
+ | 0.407 | 212.0 | 70596 | 0.3654 |
263
+ | 0.407 | 213.0 | 70929 | 0.4434 |
264
+ | 0.407 | 214.0 | 71262 | 0.3613 |
265
+ | 0.407 | 215.0 | 71595 | 0.5103 |
266
+ | 0.4069 | 216.0 | 71928 | 0.3733 |
267
+ | 0.4069 | 217.0 | 72261 | 0.4881 |
268
+ | 0.4069 | 218.0 | 72594 | 0.3375 |
269
+ | 0.4069 | 219.0 | 72927 | 0.4766 |
270
+ | 0.4069 | 220.0 | 73260 | 0.4604 |
271
 
272
 
273
  ### Framework versions