Hugging Face's logo

Shaier
/
distilbert-base-uncased-continued_training-medqa

Fill-Mask
Transformers
PyTorch
distilbert
Generated from Trainer
Inference Endpoints
Model card Files Community
Shaier commited on
Commit
db50dd3
·
1 Parent(s): da1306c

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +168 -0
README.md ADDED
@@ -0,0 +1,168 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ model-index:
5
+ - name: distilbert-base-uncased-continued_training-medqa
6
+ results: []
7
+ ---
8
+
9
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
10
+ should probably proofread and complete it, then remove this comment. -->
11
+
12
+ # distilbert-base-uncased-continued_training-medqa
13
+
14
+ This model is a fine-tuned version of [Shaier/distilbert-base-uncased-finetuned-medqa](https://huggingface.co/Shaier/distilbert-base-uncased-finetuned-medqa) on an unknown dataset.
15
+ It achieves the following results on the evaluation set:
16
+ - Loss: 0.4057
17
+
18
+ ## Model description
19
+
20
+ More information needed
21
+
22
+ ## Intended uses & limitations
23
+
24
+ More information needed
25
+
26
+ ## Training and evaluation data
27
+
28
+ More information needed
29
+
30
+ ## Training procedure
31
+
32
+ ### Training hyperparameters
33
+
34
+ The following hyperparameters were used during training:
35
+ - learning_rate: 2e-05
36
+ - train_batch_size: 64
37
+ - eval_batch_size: 64
38
+ - seed: 42
39
+ - gradient_accumulation_steps: 8
40
+ - total_train_batch_size: 512
41
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
+ - lr_scheduler_type: linear
43
+ - lr_scheduler_warmup_steps: 100
44
+ - num_epochs: 110
45
+ - mixed_precision_training: Native AMP
46
+
47
+ ### Training results
48
+
49
+ | Training Loss | Epoch | Step | Validation Loss |
50
+ |:-------------:|:-----:|:-----:|:---------------:|
51
+ | No log | 1.0 | 333 | 0.5673 |
52
+ | No log | 2.0 | 666 | 0.6109 |
53
+ | No log | 3.0 | 999 | 0.4928 |
54
+ | No log | 4.0 | 1332 | 0.5496 |
55
+ | No log | 5.0 | 1665 | 0.5017 |
56
+ | No log | 6.0 | 1998 | 0.6361 |
57
+ | No log | 7.0 | 2331 | 0.5995 |
58
+ | 0.6107 | 8.0 | 2664 | 0.6359 |
59
+ | 0.6107 | 9.0 | 2997 | 0.4778 |
60
+ | 0.6107 | 10.0 | 3330 | 0.5355 |
61
+ | 0.6107 | 11.0 | 3663 | 0.6210 |
62
+ | 0.6107 | 12.0 | 3996 | 0.4758 |
63
+ | 0.6107 | 13.0 | 4329 | 0.5468 |
64
+ | 0.6107 | 14.0 | 4662 | 0.4659 |
65
+ | 0.6107 | 15.0 | 4995 | 0.5465 |
66
+ | 0.5784 | 16.0 | 5328 | 0.3997 |
67
+ | 0.5784 | 17.0 | 5661 | 0.5352 |
68
+ | 0.5784 | 18.0 | 5994 | 0.5812 |
69
+ | 0.5784 | 19.0 | 6327 | 0.6133 |
70
+ | 0.5784 | 20.0 | 6660 | 0.5050 |
71
+ | 0.5784 | 21.0 | 6993 | 0.4377 |
72
+ | 0.5784 | 22.0 | 7326 | 0.4630 |
73
+ | 0.5784 | 23.0 | 7659 | 0.5162 |
74
+ | 0.5555 | 24.0 | 7992 | 0.4968 |
75
+ | 0.5555 | 25.0 | 8325 | 0.5336 |
76
+ | 0.5555 | 26.0 | 8658 | 0.5146 |
77
+ | 0.5555 | 27.0 | 8991 | 0.4887 |
78
+ | 0.5555 | 28.0 | 9324 | 0.5048 |
79
+ | 0.5555 | 29.0 | 9657 | 0.5092 |
80
+ | 0.5555 | 30.0 | 9990 | 0.4874 |
81
+ | 0.5555 | 31.0 | 10323 | 0.5452 |
82
+ | 0.5392 | 32.0 | 10656 | 0.5303 |
83
+ | 0.5392 | 33.0 | 10989 | 0.5932 |
84
+ | 0.5392 | 34.0 | 11322 | 0.4472 |
85
+ | 0.5392 | 35.0 | 11655 | 0.5796 |
86
+ | 0.5392 | 36.0 | 11988 | 0.4808 |
87
+ | 0.5392 | 37.0 | 12321 | 0.4756 |
88
+ | 0.5392 | 38.0 | 12654 | 0.5552 |
89
+ | 0.5392 | 39.0 | 12987 | 0.5777 |
90
+ | 0.5276 | 40.0 | 13320 | 0.5049 |
91
+ | 0.5276 | 41.0 | 13653 | 0.5141 |
92
+ | 0.5276 | 42.0 | 13986 | 0.5153 |
93
+ | 0.5276 | 43.0 | 14319 | 0.5255 |
94
+ | 0.5276 | 44.0 | 14652 | 0.4948 |
95
+ | 0.5276 | 45.0 | 14985 | 0.4542 |
96
+ | 0.5276 | 46.0 | 15318 | 0.3743 |
97
+ | 0.5276 | 47.0 | 15651 | 0.5167 |
98
+ | 0.5181 | 48.0 | 15984 | 0.5304 |
99
+ | 0.5181 | 49.0 | 16317 | 0.5459 |
100
+ | 0.5181 | 50.0 | 16650 | 0.4616 |
101
+ | 0.5181 | 51.0 | 16983 | 0.4887 |
102
+ | 0.5181 | 52.0 | 17316 | 0.4454 |
103
+ | 0.5181 | 53.0 | 17649 | 0.4665 |
104
+ | 0.5181 | 54.0 | 17982 | 0.4647 |
105
+ | 0.5181 | 55.0 | 18315 | 0.4168 |
106
+ | 0.5102 | 56.0 | 18648 | 0.5703 |
107
+ | 0.5102 | 57.0 | 18981 | 0.4930 |
108
+ | 0.5102 | 58.0 | 19314 | 0.4750 |
109
+ | 0.5102 | 59.0 | 19647 | 0.4853 |
110
+ | 0.5102 | 60.0 | 19980 | 0.4508 |
111
+ | 0.5102 | 61.0 | 20313 | 0.5869 |
112
+ | 0.5102 | 62.0 | 20646 | 0.4975 |
113
+ | 0.5102 | 63.0 | 20979 | 0.4944 |
114
+ | 0.5039 | 64.0 | 21312 | 0.4994 |
115
+ | 0.5039 | 65.0 | 21645 | 0.4257 |
116
+ | 0.5039 | 66.0 | 21978 | 0.5275 |
117
+ | 0.5039 | 67.0 | 22311 | 0.4432 |
118
+ | 0.5039 | 68.0 | 22644 | 0.5080 |
119
+ | 0.5039 | 69.0 | 22977 | 0.4262 |
120
+ | 0.5039 | 70.0 | 23310 | 0.5359 |
121
+ | 0.5039 | 71.0 | 23643 | 0.5504 |
122
+ | 0.4993 | 72.0 | 23976 | 0.4428 |
123
+ | 0.4993 | 73.0 | 24309 | 0.4275 |
124
+ | 0.4993 | 74.0 | 24642 | 0.5607 |
125
+ | 0.4993 | 75.0 | 24975 | 0.4612 |
126
+ | 0.4993 | 76.0 | 25308 | 0.5083 |
127
+ | 0.4993 | 77.0 | 25641 | 0.4803 |
128
+ | 0.4993 | 78.0 | 25974 | 0.5019 |
129
+ | 0.4993 | 79.0 | 26307 | 0.4535 |
130
+ | 0.4957 | 80.0 | 26640 | 0.5364 |
131
+ | 0.4957 | 81.0 | 26973 | 0.5502 |
132
+ | 0.4957 | 82.0 | 27306 | 0.4912 |
133
+ | 0.4957 | 83.0 | 27639 | 0.5563 |
134
+ | 0.4957 | 84.0 | 27972 | 0.4360 |
135
+ | 0.4957 | 85.0 | 28305 | 0.4962 |
136
+ | 0.4957 | 86.0 | 28638 | 0.4523 |
137
+ | 0.4957 | 87.0 | 28971 | 0.4979 |
138
+ | 0.4923 | 88.0 | 29304 | 0.4697 |
139
+ | 0.4923 | 89.0 | 29637 | 0.4730 |
140
+ | 0.4923 | 90.0 | 29970 | 0.4848 |
141
+ | 0.4923 | 91.0 | 30303 | 0.4293 |
142
+ | 0.4923 | 92.0 | 30636 | 0.4745 |
143
+ | 0.4923 | 93.0 | 30969 | 0.3710 |
144
+ | 0.4923 | 94.0 | 31302 | 0.4068 |
145
+ | 0.4923 | 95.0 | 31635 | 0.4980 |
146
+ | 0.4896 | 96.0 | 31968 | 0.4586 |
147
+ | 0.4896 | 97.0 | 32301 | 0.5152 |
148
+ | 0.4896 | 98.0 | 32634 | 0.4636 |
149
+ | 0.4896 | 99.0 | 32967 | 0.5426 |
150
+ | 0.4896 | 100.0 | 33300 | 0.4604 |
151
+ | 0.4896 | 101.0 | 33633 | 0.4925 |
152
+ | 0.4896 | 102.0 | 33966 | 0.3729 |
153
+ | 0.4896 | 103.0 | 34299 | 0.4337 |
154
+ | 0.4882 | 104.0 | 34632 | 0.5307 |
155
+ | 0.4882 | 105.0 | 34965 | 0.5480 |
156
+ | 0.4882 | 106.0 | 35298 | 0.4124 |
157
+ | 0.4882 | 107.0 | 35631 | 0.4862 |
158
+ | 0.4882 | 108.0 | 35964 | 0.4333 |
159
+ | 0.4882 | 109.0 | 36297 | 0.4443 |
160
+ | 0.4882 | 110.0 | 36630 | 0.4890 |
161
+
162
+
163
+ ### Framework versions
164
+
165
+ - Transformers 4.18.0
166
+ - Pytorch 1.11.0
167
+ - Datasets 2.3.2
168
+ - Tokenizers 0.11.0