2020-Q3-90p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.7327

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.17	8000	4.1206
4.294	0.34	16000	4.0057
4.294	0.51	24000	3.9651
4.0928	0.68	32000	3.8968
4.0928	0.85	40000	3.8837
4.0249	1.01	48000	3.8693
4.0249	1.18	56000	3.8535
3.9984	1.35	64000	3.8320
3.9984	1.52	72000	3.8221
3.9768	1.69	80000	3.8312
3.9768	1.86	88000	3.8663
3.9664	2.03	96000	3.8159
3.9664	2.2	104000	3.8216
3.9582	2.37	112000	3.8313
3.9582	2.54	120000	3.8215
3.9513	2.7	128000	3.7974
3.9513	2.87	136000	3.8077
3.9604	3.04	144000	3.7820
3.9604	3.21	152000	3.7946
3.9564	3.38	160000	3.8025
3.9564	3.55	168000	3.8236
3.9533	3.72	176000	3.8194
3.9533	3.89	184000	3.7996
3.9554	4.06	192000	3.7775
3.9554	4.23	200000	3.7879
3.9471	4.4	208000	3.8141
3.9471	4.56	216000	3.8025
3.9478	4.73	224000	3.7839
3.9478	4.9	232000	3.8026
3.9336	5.07	240000	3.8102
3.9336	5.24	248000	3.7985
3.9384	5.41	256000	3.8082
3.9384	5.58	264000	3.7966
3.9467	5.75	272000	3.7826
3.9467	5.92	280000	3.7730
3.936	6.09	288000	3.7910
3.936	6.26	296000	3.7904
3.9438	6.42	304000	3.7918
3.9438	6.59	312000	3.7773
3.9386	6.76	320000	3.7665
3.9386	6.93	328000	3.7803
3.9365	7.1	336000	3.7851
3.9365	7.27	344000	3.7787
3.946	7.44	352000	3.7683
3.946	7.61	360000	3.7781
3.9323	7.78	368000	3.7725
3.9323	7.95	376000	3.7714
3.9355	8.11	384000	3.7898
3.9355	8.28	392000	3.7781
3.9296	8.45	400000	3.7859
3.9296	8.62	408000	3.7685
3.9272	8.79	416000	3.7745
3.9272	8.96	424000	3.7547
3.9377	9.13	432000	3.7588
3.9377	9.3	440000	3.7635
3.9279	9.47	448000	3.7588
3.9279	9.64	456000	3.7531
3.9267	9.81	464000	3.7778
3.9267	9.97	472000	3.7847
3.9168	10.14	480000	3.7633
3.9168	10.31	488000	3.7564
3.9304	10.48	496000	3.7547
3.9304	10.65	504000	3.7544
3.9218	10.82	512000	3.7564
3.9218	10.99	520000	3.7775
3.9247	11.16	528000	3.7621
3.9247	11.33	536000	3.7585
3.9148	11.5	544000	3.7779
3.9148	11.66	552000	3.7580
3.9224	11.83	560000	3.7564
3.9224	12.0	568000	3.7553
3.9196	12.17	576000	3.7440
3.9196	12.34	584000	3.7556
3.9228	12.51	592000	3.7632
3.9228	12.68	600000	3.7566
3.926	12.85	608000	3.7429
3.926	13.02	616000	3.7447
3.922	13.19	624000	3.7573
3.922	13.36	632000	3.7698
3.9269	13.52	640000	3.7592
3.9269	13.69	648000	3.7665
3.918	13.86	656000	3.7588
3.918	14.03	664000	3.7394
3.9173	14.2	672000	3.7605
3.9173	14.37	680000	3.7484
3.9115	14.54	688000	3.7686
3.9115	14.71	696000	3.7632
3.9111	14.88	704000	3.7682
3.9111	15.05	712000	3.7497
3.9131	15.21	720000	3.7470
3.9131	15.38	728000	3.7567
3.9211	15.55	736000	3.7546
3.9211	15.72	744000	3.7390
3.9059	15.89	752000	3.7530
3.9059	16.06	760000	3.7728
3.9051	16.23	768000	3.7345
3.9051	16.4	776000	3.7468
3.9103	16.57	784000	3.7357
3.9103	16.74	792000	3.7434
3.9058	16.91	800000	3.7456
3.9058	17.07	808000	3.7634
3.9076	17.24	816000	3.7374
3.9076	17.41	824000	3.7534
3.9071	17.58	832000	3.7576
3.9071	17.75	840000	3.7682
3.8976	17.92	848000	3.7599
3.8976	18.09	856000	3.7436
3.9071	18.26	864000	3.7564
3.9071	18.43	872000	3.7284
3.9067	18.6	880000	3.7458
3.9067	18.77	888000	3.7448
3.914	18.93	896000	3.7490
3.914	19.1	904000	3.7290
3.9086	19.27	912000	3.7238
3.9086	19.44	920000	3.7402
3.9115	19.61	928000	3.7399
3.9115	19.78	936000	3.7504
3.905	19.95	944000	3.7206
3.905	20.12	952000	3.7351
3.9012	20.29	960000	3.7409
3.9012	20.46	968000	3.7426
3.9043	20.62	976000	3.7349
3.9043	20.79	984000	3.7376
3.9104	20.96	992000	3.7325
3.9104	21.13	1000000	3.7488
3.9025	21.3	1008000	3.7192
3.9025	21.47	1016000	3.7437
3.9131	21.64	1024000	3.7483
3.9131	21.81	1032000	3.7290
3.9048	21.98	1040000	3.7547
3.9048	22.15	1048000	3.7387
3.9011	22.32	1056000	3.7373
3.9011	22.48	1064000	3.7222
3.8988	22.65	1072000	3.7362
3.8988	22.82	1080000	3.7068
3.9022	22.99	1088000	3.7358
3.9022	23.16	1096000	3.7317
3.8916	23.33	1104000	3.7264
3.8916	23.5	1112000	3.7487
3.9079	23.67	1120000	3.7486
3.9079	23.84	1128000	3.7403
3.9037	24.01	1136000	3.7345
3.9037	24.17	1144000	3.7221
3.8932	24.34	1152000	3.7286
3.8932	24.51	1160000	3.7309
3.9055	24.68	1168000	3.7464
3.9055	24.85	1176000	3.7289
3.901	25.02	1184000	3.7245
3.901	25.19	1192000	3.7169
3.8934	25.36	1200000	3.7176
3.8934	25.53	1208000	3.7250
3.9063	25.7	1216000	3.7347
3.9063	25.87	1224000	3.7445
3.8937	26.03	1232000	3.7318
3.8937	26.2	1240000	3.7202
3.9056	26.37	1248000	3.7427
3.9056	26.54	1256000	3.7317
3.8966	26.71	1264000	3.7210
3.8966	26.88	1272000	3.7238
3.8888	27.05	1280000	3.7164
3.8888	27.22	1288000	3.7068
3.8924	27.39	1296000	3.7498
3.8924	27.56	1304000	3.7183
3.8992	27.72	1312000	3.7284
3.8992	27.89	1320000	3.7397
3.891	28.06	1328000	3.7212
3.891	28.23	1336000	3.7249
3.8947	28.4	1344000	3.7143
3.8947	28.57	1352000	3.7273
3.8938	28.74	1360000	3.7317
3.8938	28.91	1368000	3.7182
3.8848	29.08	1376000	3.7199
3.8848	29.25	1384000	3.7309
3.8907	29.42	1392000	3.7221
3.8907	29.58	1400000	3.7299
3.8956	29.75	1408000	3.7257
3.8956	29.92	1416000	3.7313
3.8915	30.09	1424000	3.7374
3.8915	30.26	1432000	3.7146
3.901	30.43	1440000	3.7190
3.901	30.6	1448000	3.7211
3.8901	30.77	1456000	3.7215
3.8901	30.94	1464000	3.7189
3.89	31.11	1472000	3.6938
3.89	31.28	1480000	3.7315
3.8981	31.44	1488000	3.7328
3.8981	31.61	1496000	3.6945
3.8866	31.78	1504000	3.7125
3.8866	31.95	1512000	3.7229
3.8943	32.12	1520000	3.7248
3.8943	32.29	1528000	3.7076
3.8978	32.46	1536000	3.7276
3.8978	32.63	1544000	3.7146
3.8917	32.8	1552000	3.7323
3.8917	32.97	1560000	3.7308
3.902	33.13	1568000	3.7290
3.902	33.3	1576000	3.7153
3.8942	33.47	1584000	3.7271
3.8942	33.64	1592000	3.7061
3.885	33.81	1600000	3.7335
3.885	33.98	1608000	3.7009
3.8916	34.15	1616000	3.7184
3.8916	34.32	1624000	3.7282
3.8941	34.49	1632000	3.7347
3.8941	34.66	1640000	3.7183
3.8923	34.83	1648000	3.7179
3.8923	34.99	1656000	3.7358
3.8959	35.16	1664000	3.7375
3.8959	35.33	1672000	3.7214
3.8864	35.5	1680000	3.6894
3.8864	35.67	1688000	3.7273
3.8882	35.84	1696000	3.7326
3.8882	36.01	1704000	3.7454
3.8897	36.18	1712000	3.7255
3.8897	36.35	1720000	3.7154
3.8846	36.52	1728000	3.7161
3.8846	36.68	1736000	3.7116
3.8768	36.85	1744000	3.6959
3.8768	37.02	1752000	3.7220
3.8972	37.19	1760000	3.7378
3.8972	37.36	1768000	3.7325
3.8849	37.53	1776000	3.7259
3.8849	37.7	1784000	3.7256
3.892	37.87	1792000	3.7198
3.892	38.04	1800000	3.7202
3.8947	38.21	1808000	3.7185
3.8947	38.38	1816000	3.7097
3.8904	38.54	1824000	3.7317
3.8904	38.71	1832000	3.7153
3.8884	38.88	1840000	3.7399
3.8884	39.05	1848000	3.7260
3.8815	39.22	1856000	3.7156
3.8815	39.39	1864000	3.7196
3.8915	39.56	1872000	3.7313
3.8915	39.73	1880000	3.7223
3.8795	39.9	1888000	3.7163
3.8795	40.07	1896000	3.7089
3.8865	40.23	1904000	3.7227
3.8865	40.4	1912000	3.7250
3.8987	40.57	1920000	3.6921
3.8987	40.74	1928000	3.7254
3.8826	40.91	1936000	3.7100
3.8826	41.08	1944000	3.7356
3.8911	41.25	1952000	3.7179
3.8911	41.42	1960000	3.7277
3.8752	41.59	1968000	3.7068
3.8752	41.76	1976000	3.7191
3.8818	41.93	1984000	3.7262
3.8818	42.09	1992000	3.7295
3.8774	42.26	2000000	3.7066
3.8774	42.43	2008000	3.6993
3.883	42.6	2016000	3.7086
3.883	42.77	2024000	3.7209
3.8885	42.94	2032000	3.7046
3.8885	43.11	2040000	3.6927
3.8909	43.28	2048000	3.7372
3.8909	43.45	2056000	3.7064
3.8904	43.62	2064000	3.7271
3.8904	43.79	2072000	3.7223
3.8877	43.95	2080000	3.7141
3.8877	44.12	2088000	3.6969
3.8886	44.29	2096000	3.7125
3.8886	44.46	2104000	3.7268
3.8832	44.63	2112000	3.7046
3.8832	44.8	2120000	3.7419
3.8812	44.97	2128000	3.7163
3.8812	45.14	2136000	3.7046
3.8852	45.31	2144000	3.7006
3.8852	45.48	2152000	3.7212
3.8864	45.64	2160000	3.7030
3.8864	45.81	2168000	3.7296
3.8867	45.98	2176000	3.7194
3.8867	46.15	2184000	3.7131
3.8856	46.32	2192000	3.7299
3.8856	46.49	2200000	3.6991
3.8843	46.66	2208000	3.7246
3.8843	46.83	2216000	3.7112
3.8923	47.0	2224000	3.7293
3.8923	47.17	2232000	3.7117
3.8905	47.34	2240000	3.7194
3.8905	47.5	2248000	3.6994
3.8818	47.67	2256000	3.7122
3.8818	47.84	2264000	3.7011
3.888	48.01	2272000	3.7052
3.888	48.18	2280000	3.7164
3.8801	48.35	2288000	3.7082
3.8801	48.52	2296000	3.7080
3.8818	48.69	2304000	3.7073
3.8818	48.86	2312000	3.7132
3.8889	49.03	2320000	3.7126
3.8889	49.19	2328000	3.7306
3.8868	49.36	2336000	3.7130
3.8868	49.53	2344000	3.7146
3.8849	49.7	2352000	3.7087
3.8849	49.87	2360000	3.7074
3.8807	50.04	2368000	3.7131
3.8807	50.21	2376000	3.7047
3.8847	50.38	2384000	3.7149
3.8847	50.55	2392000	3.7103
3.8927	50.72	2400000	3.7306

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0