Hugging Face's logo

HideOnBush
/
BERTModified-finetuned-wikitext-full

PyTorch
Generated from Trainer
Model card Files Community
HideOnBush commited on
Commit
9171c13
·
1 Parent(s): a1ac91e

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +183 -0
README.md ADDED
@@ -0,0 +1,183 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ metrics:
5
+ - precision
6
+ - recall
7
+ - f1
8
+ - accuracy
9
+ model-index:
10
+ - name: BERTModified-finetuned-wikitext-full
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # BERTModified-finetuned-wikitext-full
18
+
19
+ This model was trained from scratch on the None dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 13.7327
22
+ - Precision: 0.1676
23
+ - Recall: 0.1676
24
+ - F1: 0.1676
25
+ - Accuracy: 0.1676
26
+
27
+ ## Model description
28
+
29
+ More information needed
30
+
31
+ ## Intended uses & limitations
32
+
33
+ More information needed
34
+
35
+ ## Training and evaluation data
36
+
37
+ More information needed
38
+
39
+ ## Training procedure
40
+
41
+ ### Training hyperparameters
42
+
43
+ The following hyperparameters were used during training:
44
+ - learning_rate: 0.0001
45
+ - train_batch_size: 4
46
+ - eval_batch_size: 4
47
+ - seed: 42
48
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
+ - lr_scheduler_type: linear
50
+ - num_epochs: 120
51
+
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Accuracy | F1 | Validation Loss | Precision | Recall |
55
+ |:-------------:|:-----:|:------:|:--------:|:------:|:---------------:|:---------:|:------:|
56
+ | 22.0255 | 1.0 | 4253 | 0.0916 | 0.0916 | 20.2496 | 0.0916 | 0.0916 |
57
+ | 19.842 | 2.0 | 8506 | 0.1143 | 0.1143 | 19.0691 | 0.1143 | 0.1143 |
58
+ | 18.775 | 3.0 | 12759 | 0.1206 | 0.1206 | 18.5569 | 0.1206 | 0.1206 |
59
+ | 18.1046 | 4.0 | 17012 | 0.1251 | 0.1251 | 17.9528 | 0.1251 | 0.1251 |
60
+ | 17.683 | 5.0 | 21265 | 0.1251 | 0.1251 | 17.5635 | 0.1251 | 0.1251 |
61
+ | 17.5511 | 6.0 | 25518 | 0.1259 | 0.1259 | 16.8760 | 0.1259 | 0.1259 |
62
+ | 16.6864 | 7.0 | 29771 | 0.1335 | 0.1335 | 16.4200 | 0.1335 | 0.1335 |
63
+ | 16.0819 | 8.0 | 34024 | 0.1314 | 0.1314 | 16.0207 | 0.1314 | 0.1314 |
64
+ | 15.7008 | 9.0 | 38277 | 0.1378 | 0.1378 | 15.7501 | 0.1378 | 0.1378 |
65
+ | 15.4118 | 10.0 | 42530 | 0.1388 | 0.1388 | 15.5762 | 0.1388 | 0.1388 |
66
+ | 15.1596 | 11.0 | 46783 | 0.1449 | 0.1449 | 15.3382 | 0.1449 | 0.1449 |
67
+ | 14.9354 | 12.0 | 51036 | 0.1452 | 0.1452 | 15.2556 | 0.1452 | 0.1452 |
68
+ | 14.7266 | 13.0 | 55289 | 0.1464 | 0.1464 | 15.0213 | 0.1464 | 0.1464 |
69
+ | 14.5535 | 14.0 | 59542 | 0.1442 | 0.1442 | 14.9560 | 0.1442 | 0.1442 |
70
+ | 14.3903 | 15.0 | 63795 | 0.1538 | 0.1538 | 14.8365 | 0.1538 | 0.1538 |
71
+ | 14.2298 | 16.0 | 68048 | 0.1523 | 0.1523 | 14.8610 | 0.1523 | 0.1523 |
72
+ | 14.0871 | 17.0 | 72301 | 0.1536 | 0.1536 | 14.6533 | 0.1536 | 0.1536 |
73
+ | 13.9559 | 18.0 | 76554 | 0.1613 | 0.1613 | 14.6200 | 0.1613 | 0.1613 |
74
+ | 13.8601 | 19.0 | 80807 | 0.1624 | 0.1624 | 14.5995 | 0.1624 | 0.1624 |
75
+ | 13.7507 | 20.0 | 85060 | 0.1539 | 0.1539 | 14.4666 | 0.1539 | 0.1539 |
76
+ | 13.6544 | 21.0 | 89313 | 0.1575 | 0.1575 | 14.5517 | 0.1575 | 0.1575 |
77
+ | 13.5541 | 22.0 | 93566 | 0.1590 | 0.1590 | 14.5004 | 0.1590 | 0.1590 |
78
+ | 13.4733 | 23.0 | 97819 | 0.1648 | 0.1648 | 14.4233 | 0.1648 | 0.1648 |
79
+ | 13.3851 | 24.0 | 102072 | 0.1561 | 0.1561 | 14.3381 | 0.1561 | 0.1561 |
80
+ | 13.3094 | 25.0 | 106325 | 0.1630 | 0.1630 | 14.3428 | 0.1630 | 0.1630 |
81
+ | 13.2396 | 26.0 | 110578 | 0.1602 | 0.1602 | 14.3448 | 0.1602 | 0.1602 |
82
+ | 13.1548 | 27.0 | 114831 | 0.1603 | 0.1603 | 14.4059 | 0.1603 | 0.1603 |
83
+ | 13.0713 | 28.0 | 119084 | 0.1647 | 0.1647 | 14.2224 | 0.1647 | 0.1647 |
84
+ | 13.0317 | 29.0 | 123337 | 0.1619 | 0.1619 | 14.1962 | 0.1619 | 0.1619 |
85
+ | 12.9748 | 30.0 | 127590 | 0.1633 | 0.1633 | 14.2379 | 0.1633 | 0.1633 |
86
+ | 12.9308 | 31.0 | 131843 | 0.1648 | 0.1648 | 14.3591 | 0.1648 | 0.1648 |
87
+ | 12.8504 | 32.0 | 136096 | 0.1612 | 0.1612 | 14.1515 | 0.1612 | 0.1612 |
88
+ | 12.8249 | 33.0 | 140349 | 0.1625 | 0.1625 | 14.2247 | 0.1625 | 0.1625 |
89
+ | 12.76 | 34.0 | 144602 | 0.1588 | 0.1588 | 14.3861 | 0.1588 | 0.1588 |
90
+ | 12.7458 | 35.0 | 148855 | 0.1677 | 0.1677 | 14.1783 | 0.1677 | 0.1677 |
91
+ | 12.6804 | 36.0 | 153108 | 0.1639 | 0.1639 | 14.0968 | 0.1639 | 0.1639 |
92
+ | 12.6653 | 37.0 | 157361 | 0.1604 | 0.1604 | 13.9660 | 0.1604 | 0.1604 |
93
+ | 12.6191 | 38.0 | 161614 | 0.1680 | 0.1680 | 14.1075 | 0.1680 | 0.1680 |
94
+ | 12.5721 | 39.0 | 165867 | 0.1664 | 0.1664 | 13.9293 | 0.1664 | 0.1664 |
95
+ | 12.5647 | 40.0 | 170120 | 0.1693 | 0.1693 | 14.0956 | 0.1693 | 0.1693 |
96
+ | 12.5205 | 41.0 | 174373 | 0.1680 | 0.1680 | 14.1204 | 0.1680 | 0.1680 |
97
+ | 12.4846 | 42.0 | 178626 | 0.1655 | 0.1655 | 14.0288 | 0.1655 | 0.1655 |
98
+ | 12.4415 | 43.0 | 182879 | 0.1659 | 0.1659 | 14.1088 | 0.1659 | 0.1659 |
99
+ | 12.4219 | 44.0 | 187132 | 0.1661 | 0.1661 | 14.1751 | 0.1661 | 0.1661 |
100
+ | 12.4161 | 45.0 | 191385 | 0.1691 | 0.1691 | 14.0942 | 0.1691 | 0.1691 |
101
+ | 12.3982 | 46.0 | 195638 | 0.1607 | 0.1607 | 14.1141 | 0.1607 | 0.1607 |
102
+ | 12.3699 | 47.0 | 199891 | 0.1660 | 0.1660 | 14.0529 | 0.1660 | 0.1660 |
103
+ | 12.3591 | 48.0 | 204144 | 0.1693 | 0.1693 | 13.8278 | 0.1693 | 0.1693 |
104
+ | 12.3295 | 49.0 | 208397 | 0.1662 | 0.1662 | 14.2327 | 0.1662 | 0.1662 |
105
+ | 12.3284 | 50.0 | 212650 | 0.1685 | 0.1685 | 14.1071 | 0.1685 | 0.1685 |
106
+ | 12.5499 | 51.0 | 216903 | 0.1703 | 0.1703 | 14.0200 | 0.1703 | 0.1703 |
107
+ | 12.5456 | 52.0 | 221156 | 0.1679 | 0.1679 | 14.1215 | 0.1679 | 0.1679 |
108
+ | 12.5275 | 53.0 | 225409 | 0.1645 | 0.1645 | 14.0103 | 0.1645 | 0.1645 |
109
+ | 12.4778 | 54.0 | 229662 | 0.1669 | 0.1669 | 14.0708 | 0.1669 | 0.1669 |
110
+ | 12.4607 | 55.0 | 233915 | 0.1681 | 0.1681 | 13.9272 | 0.1681 | 0.1681 |
111
+ | 12.3859 | 56.0 | 238168 | 0.1653 | 0.1653 | 14.0765 | 0.1653 | 0.1653 |
112
+ | 12.3655 | 57.0 | 242421 | 0.1667 | 0.1667 | 13.9649 | 0.1667 | 0.1667 |
113
+ | 12.3407 | 58.0 | 246674 | 0.1671 | 0.1671 | 13.9990 | 0.1671 | 0.1671 |
114
+ | 12.3088 | 59.0 | 250927 | 0.1656 | 0.1656 | 14.1460 | 0.1656 | 0.1656 |
115
+ | 12.2647 | 60.0 | 255180 | 0.1634 | 0.1634 | 14.0701 | 0.1634 | 0.1634 |
116
+ | 12.2248 | 61.0 | 259433 | 0.1647 | 0.1647 | 14.1750 | 0.1647 | 0.1647 |
117
+ | 12.1574 | 62.0 | 263686 | 0.1664 | 0.1664 | 14.0358 | 0.1664 | 0.1664 |
118
+ | 12.1868 | 63.0 | 267939 | 0.1655 | 0.1655 | 14.1684 | 0.1655 | 0.1655 |
119
+ | 12.1522 | 64.0 | 272192 | 0.1629 | 0.1629 | 13.8736 | 0.1629 | 0.1629 |
120
+ | 12.1013 | 65.0 | 276445 | 0.1679 | 0.1679 | 14.0409 | 0.1679 | 0.1679 |
121
+ | 12.0726 | 66.0 | 280698 | 0.1664 | 0.1664 | 14.1370 | 0.1664 | 0.1664 |
122
+ | 12.0498 | 67.0 | 284951 | 0.1636 | 0.1636 | 14.3254 | 0.1636 | 0.1636 |
123
+ | 12.016 | 68.0 | 289204 | 0.1656 | 0.1656 | 13.9675 | 0.1656 | 0.1656 |
124
+ | 12.0105 | 69.0 | 293457 | 0.1683 | 0.1683 | 13.9938 | 0.1683 | 0.1683 |
125
+ | 11.9617 | 70.0 | 297710 | 0.1691 | 0.1691 | 13.8582 | 0.1691 | 0.1691 |
126
+ | 11.9515 | 71.0 | 301963 | 0.1633 | 0.1633 | 14.1426 | 0.1633 | 0.1633 |
127
+ | 11.9394 | 72.0 | 306216 | 0.1666 | 0.1666 | 14.1693 | 0.1666 | 0.1666 |
128
+ | 11.8926 | 73.0 | 310469 | 0.1686 | 0.1686 | 13.8987 | 0.1686 | 0.1686 |
129
+ | 11.8816 | 74.0 | 314722 | 0.1703 | 0.1703 | 13.8562 | 0.1703 | 0.1703 |
130
+ | 11.8537 | 75.0 | 318975 | 0.1664 | 0.1664 | 13.8956 | 0.1664 | 0.1664 |
131
+ | 11.8447 | 76.0 | 323228 | 0.1672 | 0.1672 | 13.9239 | 0.1672 | 0.1672 |
132
+ | 11.8166 | 77.0 | 327481 | 0.1712 | 0.1712 | 13.9084 | 0.1712 | 0.1712 |
133
+ | 11.7861 | 78.0 | 331734 | 0.1721 | 0.1721 | 13.8685 | 0.1721 | 0.1721 |
134
+ | 11.7681 | 79.0 | 335987 | 0.1679 | 0.1679 | 13.8594 | 0.1679 | 0.1679 |
135
+ | 11.7506 | 80.0 | 340240 | 0.1655 | 0.1655 | 13.8944 | 0.1655 | 0.1655 |
136
+ | 11.7237 | 81.0 | 344493 | 0.1678 | 0.1678 | 13.8826 | 0.1678 | 0.1678 |
137
+ | 11.7115 | 82.0 | 348746 | 0.1650 | 0.1650 | 13.7157 | 0.1650 | 0.1650 |
138
+ | 11.6859 | 83.0 | 352999 | 0.1713 | 0.1713 | 14.0802 | 0.1713 | 0.1713 |
139
+ | 11.6991 | 84.0 | 357252 | 0.1722 | 0.1722 | 13.8076 | 0.1722 | 0.1722 |
140
+ | 11.6674 | 85.0 | 361505 | 0.1712 | 0.1712 | 13.9484 | 0.1712 | 0.1712 |
141
+ | 11.6401 | 86.0 | 365758 | 0.1718 | 0.1718 | 13.8485 | 0.1718 | 0.1718 |
142
+ | 11.6477 | 87.0 | 370011 | 0.1701 | 0.1701 | 13.7326 | 0.1701 | 0.1701 |
143
+ | 11.6111 | 88.0 | 374264 | 0.1695 | 0.1695 | 14.0255 | 0.1695 | 0.1695 |
144
+ | 11.6031 | 89.0 | 378517 | 0.1704 | 0.1704 | 13.7595 | 0.1704 | 0.1704 |
145
+ | 11.5898 | 90.0 | 382770 | 0.1677 | 0.1677 | 14.0630 | 0.1677 | 0.1677 |
146
+ | 11.5808 | 91.0 | 387023 | 0.1743 | 0.1743 | 13.9293 | 0.1743 | 0.1743 |
147
+ | 11.5664 | 92.0 | 391276 | 0.1712 | 0.1712 | 13.9230 | 0.1712 | 0.1712 |
148
+ | 11.5692 | 93.0 | 395529 | 0.1694 | 0.1694 | 13.9230 | 0.1694 | 0.1694 |
149
+ | 11.5578 | 94.0 | 399782 | 0.1713 | 0.1713 | 13.9748 | 0.1713 | 0.1713 |
150
+ | 11.5067 | 95.0 | 404035 | 0.1770 | 0.1770 | 13.8655 | 0.1770 | 0.1770 |
151
+ | 11.5176 | 96.0 | 408288 | 0.1740 | 0.1740 | 14.0284 | 0.1740 | 0.1740 |
152
+ | 11.5408 | 97.0 | 412541 | 0.1718 | 0.1718 | 13.8855 | 0.1718 | 0.1718 |
153
+ | 11.4876 | 98.0 | 416794 | 0.1746 | 0.1746 | 13.6721 | 0.1746 | 0.1746 |
154
+ | 11.5056 | 99.0 | 421047 | 0.1734 | 0.1734 | 13.8167 | 0.1734 | 0.1734 |
155
+ | 11.5185 | 100.0 | 425300 | 0.1716 | 0.1716 | 13.8581 | 0.1716 | 0.1716 |
156
+ | 11.5619 | 101.0 | 429553 | 13.9219 | 0.1687 | 0.1687 | 0.1687 | 0.1687 |
157
+ | 11.5759 | 102.0 | 433806 | 14.1460 | 0.1663 | 0.1663 | 0.1663 | 0.1663 |
158
+ | 11.5661 | 103.0 | 438059 | 13.8979 | 0.1724 | 0.1724 | 0.1724 | 0.1724 |
159
+ | 11.5127 | 104.0 | 442312 | 13.7580 | 0.1727 | 0.1727 | 0.1727 | 0.1727 |
160
+ | 11.5476 | 105.0 | 446565 | 14.1029 | 0.1675 | 0.1675 | 0.1675 | 0.1675 |
161
+ | 11.5309 | 106.0 | 450818 | 14.0572 | 0.1704 | 0.1704 | 0.1704 | 0.1704 |
162
+ | 11.507 | 107.0 | 455071 | 13.8523 | 0.1734 | 0.1734 | 0.1734 | 0.1734 |
163
+ | 11.5251 | 108.0 | 459324 | 13.9018 | 0.1701 | 0.1701 | 0.1701 | 0.1701 |
164
+ | 11.4931 | 109.0 | 463577 | 13.7898 | 0.1786 | 0.1786 | 0.1786 | 0.1786 |
165
+ | 11.5107 | 110.0 | 467830 | 13.8426 | 0.1764 | 0.1764 | 0.1764 | 0.1764 |
166
+ | 11.4665 | 111.0 | 472083 | 13.9605 | 0.1705 | 0.1705 | 0.1705 | 0.1705 |
167
+ | 11.4901 | 112.0 | 476336 | 13.8094 | 0.1743 | 0.1743 | 0.1743 | 0.1743 |
168
+ | 11.4702 | 113.0 | 480589 | 13.9445 | 0.1664 | 0.1664 | 0.1664 | 0.1664 |
169
+ | 11.4617 | 114.0 | 484842 | 13.8887 | 0.1707 | 0.1707 | 0.1707 | 0.1707 |
170
+ | 11.4458 | 115.0 | 489095 | 13.8951 | 0.1701 | 0.1701 | 0.1701 | 0.1701 |
171
+ | 11.4361 | 116.0 | 493348 | 13.7645 | 0.1705 | 0.1705 | 0.1705 | 0.1705 |
172
+ | 11.4176 | 117.0 | 497601 | 13.7752 | 0.1734 | 0.1734 | 0.1734 | 0.1734 |
173
+ | 11.4359 | 118.0 | 501854 | 13.9083 | 0.1714 | 0.1714 | 0.1714 | 0.1714 |
174
+ | 11.409 | 119.0 | 506107 | 13.9772 | 0.1668 | 0.1668 | 0.1668 | 0.1668 |
175
+ | 11.4238 | 120.0 | 510360 | 13.7327 | 0.1676 | 0.1676 | 0.1676 | 0.1676 |
176
+
177
+
178
+ ### Framework versions
179
+
180
+ - Transformers 4.24.0
181
+ - Pytorch 2.0.0
182
+ - Datasets 2.6.1
183
+ - Tokenizers 0.13.1