2020-Q3-75p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1267

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.07	8000	3.4513
3.6645	0.14	16000	3.3790
3.6645	0.2	24000	3.3226
3.4678	0.27	32000	3.2901
3.4678	0.34	40000	3.2590
3.4118	0.41	48000	3.2534
3.4118	0.47	56000	3.2357
3.3843	0.54	64000	3.2228
3.3843	0.61	72000	3.2331
3.3633	0.68	80000	3.2047
3.3633	0.74	88000	3.2138
3.3474	0.81	96000	3.2050
3.3474	0.88	104000	3.2051
3.3414	0.95	112000	3.1930
3.3414	1.01	120000	3.2002
3.335	1.08	128000	3.1920
3.335	1.15	136000	3.1914
3.3283	1.22	144000	3.1853
3.3283	1.28	152000	3.1825
3.3276	1.35	160000	3.1827
3.3276	1.42	168000	3.1756
3.323	1.49	176000	3.1865
3.323	1.56	184000	3.1749
3.3275	1.62	192000	3.1782
3.3275	1.69	200000	3.1676
3.3309	1.76	208000	3.1832
3.3309	1.83	216000	3.1744
3.3166	1.89	224000	3.1645
3.3166	1.96	232000	3.1770
3.3206	2.03	240000	3.1656
3.3206	2.1	248000	3.1561
3.3228	2.16	256000	3.1665
3.3228	2.23	264000	3.1657
3.3208	2.3	272000	3.1693
3.3208	2.37	280000	3.1778
3.3106	2.43	288000	3.1760
3.3106	2.5	296000	3.1664
3.3189	2.57	304000	3.1677
3.3189	2.64	312000	3.1600
3.319	2.7	320000	3.1570
3.319	2.77	328000	3.1699
3.3236	2.84	336000	3.1578
3.3236	2.91	344000	3.1665
3.3205	2.98	352000	3.1558
3.3205	3.04	360000	3.1678
3.3114	3.11	368000	3.1597
3.3114	3.18	376000	3.1618
3.3067	3.25	384000	3.1584
3.3067	3.31	392000	3.1597
3.314	3.38	400000	3.1565
3.314	3.45	408000	3.1612
3.3183	3.52	416000	3.1637
3.3183	3.58	424000	3.1569
3.318	3.65	432000	3.1576
3.318	3.72	440000	3.1639
3.3114	3.79	448000	3.1460
3.3114	3.85	456000	3.1611
3.3068	3.92	464000	3.1587
3.3068	3.99	472000	3.1542
3.3166	4.06	480000	3.1422
3.3166	4.12	488000	3.1604
3.3057	4.19	496000	3.1587
3.3057	4.26	504000	3.1576
3.3176	4.33	512000	3.1602
3.3176	4.4	520000	3.1544
3.3126	4.46	528000	3.1478
3.3126	4.53	536000	3.1520
3.3044	4.6	544000	3.1581
3.3044	4.67	552000	3.1625
3.3118	4.73	560000	3.1510
3.3118	4.8	568000	3.1548
3.3085	4.87	576000	3.1539
3.3085	4.94	584000	3.1503
3.3014	5.0	592000	3.1504
3.3014	5.07	600000	3.1534
3.3115	5.14	608000	3.1551
3.3115	5.21	616000	3.1493
3.3079	5.27	624000	3.1427
3.3079	5.34	632000	3.1500
3.3138	5.41	640000	3.1546
3.3138	5.48	648000	3.1482
3.3096	5.54	656000	3.1346
3.3096	5.61	664000	3.1328
3.3121	5.68	672000	3.1500
3.3121	5.75	680000	3.1312
3.3195	5.82	688000	3.1440
3.3195	5.88	696000	3.1191
3.3091	5.95	704000	3.1397
3.3091	6.02	712000	3.1485
3.3089	6.09	720000	3.1340
3.3089	6.15	728000	3.1385
3.3062	6.22	736000	3.1358
3.3062	6.29	744000	3.1296
3.3102	6.36	752000	3.1260
3.3102	6.42	760000	3.1428
3.3088	6.49	768000	3.1372
3.3088	6.56	776000	3.1404
3.3096	6.63	784000	3.1362
3.3096	6.69	792000	3.1408
3.3079	6.76	800000	3.1350
3.3079	6.83	808000	3.1461
3.3099	6.9	816000	3.1420
3.3099	6.96	824000	3.1213
3.3015	7.03	832000	3.1366
3.3015	7.1	840000	3.1401
3.3045	7.17	848000	3.1295
3.3045	7.24	856000	3.1323
3.3085	7.3	864000	3.1368
3.3085	7.37	872000	3.1275
3.3061	7.44	880000	3.1326
3.3061	7.51	888000	3.1377
3.309	7.57	896000	3.1407
3.309	7.64	904000	3.1324
3.3024	7.71	912000	3.1187
3.3024	7.78	920000	3.1514
3.2955	7.84	928000	3.1351
3.2955	7.91	936000	3.1308
3.3122	7.98	944000	3.1405
3.3122	8.05	952000	3.1291
3.304	8.11	960000	3.1244
3.304	8.18	968000	3.1409
3.3046	8.25	976000	3.1355
3.3046	8.32	984000	3.1416
3.3022	8.38	992000	3.1258
3.3022	8.45	1000000	3.1332
3.3004	8.52	1008000	3.1430
3.3004	8.59	1016000	3.1282
3.3045	8.66	1024000	3.1287
3.3045	8.72	1032000	3.1368
3.3047	8.79	1040000	3.1362
3.3047	8.86	1048000	3.1268
3.3044	8.93	1056000	3.1329
3.3044	8.99	1064000	3.1245
3.2961	9.06	1072000	3.1271
3.2961	9.13	1080000	3.1300
3.2999	9.2	1088000	3.1369
3.2999	9.26	1096000	3.1425
3.3012	9.33	1104000	3.1213
3.3012	9.4	1112000	3.1285
3.3008	9.47	1120000	3.1353
3.3008	9.53	1128000	3.1367
3.3028	9.6	1136000	3.1294
3.3028	9.67	1144000	3.1340
3.3043	9.74	1152000	3.1330
3.3043	9.8	1160000	3.1380
3.2976	9.87	1168000	3.1198
3.2976	9.94	1176000	3.1290
3.3048	10.01	1184000	3.1458
3.3048	10.08	1192000	3.1274
3.3038	10.14	1200000	3.1181
3.3038	10.21	1208000	3.1279
3.3066	10.28	1216000	3.1230
3.3066	10.35	1224000	3.1320
3.3019	10.41	1232000	3.1203
3.3019	10.48	1240000	3.1349
3.3037	10.55	1248000	3.1323
3.3037	10.62	1256000	3.1343
3.2868	10.68	1264000	3.1262
3.2868	10.75	1272000	3.1266
3.3033	10.82	1280000	3.1283
3.3033	10.89	1288000	3.1290
3.2984	10.95	1296000	3.1177
3.2984	11.02	1304000	3.1234
3.2982	11.09	1312000	3.1310
3.2982	11.16	1320000	3.1409
3.303	11.23	1328000	3.1330
3.303	11.29	1336000	3.1281
3.2976	11.36	1344000	3.1286
3.2976	11.43	1352000	3.1283
3.2923	11.5	1360000	3.1146
3.2923	11.56	1368000	3.1387
3.2988	11.63	1376000	3.1278
3.2988	11.7	1384000	3.1225
3.299	11.77	1392000	3.1341
3.299	11.83	1400000	3.1211
3.2993	11.9	1408000	3.1026
3.2993	11.97	1416000	3.1223
3.2942	12.04	1424000	3.1200
3.2942	12.1	1432000	3.1246
3.3062	12.17	1440000	3.1325
3.3062	12.24	1448000	3.1388
3.297	12.31	1456000	3.1371
3.297	12.37	1464000	3.1272
3.3033	12.44	1472000	3.1231
3.3033	12.51	1480000	3.1316
3.291	12.58	1488000	3.1393
3.291	12.65	1496000	3.1269
3.3054	12.71	1504000	3.1363
3.3054	12.78	1512000	3.1249
3.2908	12.85	1520000	3.1310
3.2908	12.92	1528000	3.1213
3.2987	12.98	1536000	3.1223
3.2987	13.05	1544000	3.1134
3.2965	13.12	1552000	3.1168
3.2965	13.19	1560000	3.1230
3.2931	13.25	1568000	3.1132
3.2931	13.32	1576000	3.1196
3.301	13.39	1584000	3.1287
3.301	13.46	1592000	3.1145
3.3004	13.52	1600000	3.1291
3.3004	13.59	1608000	3.1145
3.2992	13.66	1616000	3.1292
3.2992	13.73	1624000	3.1248
3.2974	13.79	1632000	3.1315
3.2974	13.86	1640000	3.1112
3.2993	13.93	1648000	3.1217
3.2993	14.0	1656000	3.1362
3.2934	14.07	1664000	3.1199
3.2934	14.13	1672000	3.1276
3.2964	14.2	1680000	3.1164
3.2964	14.27	1688000	3.1172
3.305	14.34	1696000	3.1320
3.305	14.4	1704000	3.1269
3.3022	14.47	1712000	3.1107
3.3022	14.54	1720000	3.1097
3.2969	14.61	1728000	3.1176
3.2969	14.67	1736000	3.1282
3.2976	14.74	1744000	3.1195
3.2976	14.81	1752000	3.1154
3.3004	14.88	1760000	3.1147
3.3004	14.94	1768000	3.1094
3.2908	15.01	1776000	3.1313
3.2908	15.08	1784000	3.1280
3.2896	15.15	1792000	3.1304
3.2896	15.21	1800000	3.1329
3.3061	15.28	1808000	3.1198
3.3061	15.35	1816000	3.1258
3.3056	15.42	1824000	3.1253
3.3056	15.49	1832000	3.1200
3.2921	15.55	1840000	3.1384
3.2921	15.62	1848000	3.1225
3.2895	15.69	1856000	3.1284
3.2895	15.76	1864000	3.1201
3.293	15.82	1872000	3.1256
3.293	15.89	1880000	3.1166
3.2963	15.96	1888000	3.1218
3.2963	16.03	1896000	3.1193
3.2908	16.09	1904000	3.1204
3.2908	16.16	1912000	3.1325
3.3039	16.23	1920000	3.1091
3.3039	16.3	1928000	3.1250
3.3011	16.36	1936000	3.1217
3.3011	16.43	1944000	3.1208
3.3003	16.5	1952000	3.1109
3.3003	16.57	1960000	3.1252
3.3012	16.63	1968000	3.1123
3.3012	16.7	1976000	3.1213
3.2885	16.77	1984000	3.1219
3.2885	16.84	1992000	3.1254
3.2982	16.91	2000000	3.1260
3.2982	16.97	2008000	3.1167
3.2962	17.04	2016000	3.1082
3.2962	17.11	2024000	3.1204
3.2889	17.18	2032000	3.1236
3.2889	17.24	2040000	3.1325
3.2892	17.31	2048000	3.1200
3.2892	17.38	2056000	3.1231
3.3028	17.45	2064000	3.1202
3.3028	17.51	2072000	3.1189
3.2889	17.58	2080000	3.1337
3.2889	17.65	2088000	3.1156
3.2985	17.72	2096000	3.1258
3.2985	17.78	2104000	3.1358
3.2949	17.85	2112000	3.1271
3.2949	17.92	2120000	3.1250
3.2987	17.99	2128000	3.1244
3.2987	18.05	2136000	3.1221
3.2884	18.12	2144000	3.1198
3.2884	18.19	2152000	3.1170
3.2918	18.26	2160000	3.1159
3.2918	18.33	2168000	3.1153
3.2995	18.39	2176000	3.1203
3.2995	18.46	2184000	3.1107
3.3003	18.53	2192000	3.1212
3.3003	18.6	2200000	3.1330
3.2921	18.66	2208000	3.1160
3.2921	18.73	2216000	3.1192
3.293	18.8	2224000	3.1164
3.293	18.87	2232000	3.1225
3.2969	18.93	2240000	3.1243
3.2969	19.0	2248000	3.1152
3.2891	19.07	2256000	3.1323
3.2891	19.14	2264000	3.1077
3.2903	19.2	2272000	3.1348
3.2903	19.27	2280000	3.1202
3.2986	19.34	2288000	3.1220
3.2986	19.41	2296000	3.1236
3.293	19.47	2304000	3.1224
3.293	19.54	2312000	3.1247
3.299	19.61	2320000	3.1235
3.299	19.68	2328000	3.1201
3.2898	19.75	2336000	3.1163
3.2898	19.81	2344000	3.1289
3.2956	19.88	2352000	3.1198
3.2956	19.95	2360000	3.1251
3.2926	20.02	2368000	3.1087
3.2926	20.08	2376000	3.1097
3.2958	20.15	2384000	3.1262
3.2958	20.22	2392000	3.1308
3.2862	20.29	2400000	3.1129

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0