2020-Q2-75p-filtered-random

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9264

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.07	8000	2.2326
2.407	0.13	16000	2.1532
2.407	0.2	24000	2.1195
2.23	0.27	32000	2.0944
2.23	0.34	40000	2.0626
2.1798	0.4	48000	2.0517
2.1798	0.47	56000	2.0355
2.1621	0.54	64000	2.0469
2.1621	0.61	72000	2.0306
2.1419	0.67	80000	2.0182
2.1419	0.74	88000	2.0107
2.1264	0.81	96000	2.0096
2.1264	0.88	104000	2.0104
2.1203	0.94	112000	2.0037
2.1203	1.01	120000	2.0078
2.1116	1.08	128000	1.9965
2.1116	1.15	136000	2.0025
2.1041	1.21	144000	1.9929
2.1041	1.28	152000	1.9870
2.1058	1.35	160000	1.9895
2.1058	1.41	168000	1.9795
2.1064	1.48	176000	1.9893
2.1064	1.55	184000	1.9877
2.098	1.62	192000	1.9920
2.098	1.68	200000	1.9801
2.0916	1.75	208000	1.9778
2.0916	1.82	216000	1.9745
2.0951	1.89	224000	1.9831
2.0951	1.95	232000	1.9749
2.092	2.02	240000	1.9754
2.092	2.09	248000	1.9794
2.0968	2.16	256000	1.9675
2.0968	2.22	264000	1.9710
2.0942	2.29	272000	1.9712
2.0942	2.36	280000	1.9662
2.0929	2.43	288000	1.9672
2.0929	2.49	296000	1.9830
2.092	2.56	304000	1.9804
2.092	2.63	312000	1.9661
2.0886	2.69	320000	1.9668
2.0886	2.76	328000	1.9643
2.0883	2.83	336000	1.9710
2.0883	2.9	344000	1.9678
2.0937	2.96	352000	1.9737
2.0937	3.03	360000	1.9638
2.0899	3.1	368000	1.9599
2.0899	3.17	376000	1.9570
2.0839	3.23	384000	1.9688
2.0839	3.3	392000	1.9613
2.0862	3.37	400000	1.9686
2.0862	3.44	408000	1.9690
2.0844	3.5	416000	1.9665
2.0844	3.57	424000	1.9512
2.0802	3.64	432000	1.9652
2.0802	3.71	440000	1.9594
2.0882	3.77	448000	1.9543
2.0882	3.84	456000	1.9635
2.0794	3.91	464000	1.9618
2.0794	3.97	472000	1.9617
2.0848	4.04	480000	1.9597
2.0848	4.11	488000	1.9586
2.0814	4.18	496000	1.9587
2.0814	4.24	504000	1.9510
2.0765	4.31	512000	1.9643
2.0765	4.38	520000	1.9586
2.0887	4.45	528000	1.9476
2.0887	4.51	536000	1.9539
2.0857	4.58	544000	1.9538
2.0857	4.65	552000	1.9528
2.0798	4.72	560000	1.9586
2.0798	4.78	568000	1.9660
2.0752	4.85	576000	1.9639
2.0752	4.92	584000	1.9505
2.0771	4.99	592000	1.9551
2.0771	5.05	600000	1.9495
2.0772	5.12	608000	1.9536
2.0772	5.19	616000	1.9567
2.0836	5.25	624000	1.9534
2.0836	5.32	632000	1.9663
2.0851	5.39	640000	1.9535
2.0851	5.46	648000	1.9554
2.0842	5.52	656000	1.9539
2.0842	5.59	664000	1.9589
2.088	5.66	672000	1.9572
2.088	5.73	680000	1.9603
2.075	5.79	688000	1.9639
2.075	5.86	696000	1.9537
2.077	5.93	704000	1.9612
2.077	6.0	712000	1.9571
2.0692	6.06	720000	1.9545
2.0692	6.13	728000	1.9494
2.087	6.2	736000	1.9555
2.087	6.27	744000	1.9566
2.0783	6.33	752000	1.9447
2.0783	6.4	760000	1.9518
2.0771	6.47	768000	1.9429
2.0771	6.53	776000	1.9603
2.0794	6.6	784000	1.9503
2.0794	6.67	792000	1.9572
2.0777	6.74	800000	1.9607
2.0777	6.8	808000	1.9525
2.0725	6.87	816000	1.9495
2.0725	6.94	824000	1.9517
2.0863	7.01	832000	1.9523
2.0863	7.07	840000	1.9441
2.0735	7.14	848000	1.9430
2.0735	7.21	856000	1.9517
2.0808	7.28	864000	1.9442
2.0808	7.34	872000	1.9414
2.0756	7.41	880000	1.9439
2.0756	7.48	888000	1.9428
2.0799	7.55	896000	1.9472
2.0799	7.61	904000	1.9426
2.0717	7.68	912000	1.9461
2.0717	7.75	920000	1.9583
2.0799	7.81	928000	1.9433
2.0799	7.88	936000	1.9442
2.0704	7.95	944000	1.9396
2.0704	8.02	952000	1.9409
2.0785	8.08	960000	1.9520
2.0785	8.15	968000	1.9409
2.0761	8.22	976000	1.9469
2.0761	8.29	984000	1.9372
2.0739	8.35	992000	1.9385
2.0739	8.42	1000000	1.9540
2.0761	8.49	1008000	1.9488
2.0761	8.56	1016000	1.9464
2.0725	8.62	1024000	1.9466
2.0725	8.69	1032000	1.9460
2.0704	8.76	1040000	1.9449
2.0704	8.83	1048000	1.9493
2.0734	8.89	1056000	1.9463
2.0734	8.96	1064000	1.9403
2.0744	9.03	1072000	1.9467
2.0744	9.09	1080000	1.9406
2.0776	9.16	1088000	1.9492
2.0776	9.23	1096000	1.9433
2.068	9.3	1104000	1.9450
2.068	9.36	1112000	1.9473
2.0755	9.43	1120000	1.9459
2.0755	9.5	1128000	1.9563
2.0783	9.57	1136000	1.9369
2.0783	9.63	1144000	1.9461
2.0776	9.7	1152000	1.9494
2.0776	9.77	1160000	1.9312
2.0757	9.84	1168000	1.9452
2.0757	9.9	1176000	1.9425
2.0776	9.97	1184000	1.9536
2.0776	10.04	1192000	1.9351
2.0769	10.11	1200000	1.9301
2.0769	10.17	1208000	1.9464
2.071	10.24	1216000	1.9410
2.071	10.31	1224000	1.9321
2.0702	10.37	1232000	1.9406
2.0702	10.44	1240000	1.9525
2.0716	10.51	1248000	1.9418
2.0716	10.58	1256000	1.9373
2.0753	10.64	1264000	1.9363
2.0753	10.71	1272000	1.9504
2.0757	10.78	1280000	1.9376
2.0757	10.85	1288000	1.9351
2.0656	10.91	1296000	1.9445
2.0656	10.98	1304000	1.9282
2.0732	11.05	1312000	1.9437
2.0732	11.12	1320000	1.9501
2.0756	11.18	1328000	1.9379
2.0756	11.25	1336000	1.9430
2.0885	11.32	1344000	1.9392
2.0885	11.39	1352000	1.9344
2.0758	11.45	1360000	1.9364
2.0758	11.52	1368000	1.9404
2.0693	11.59	1376000	1.9347
2.0693	11.65	1384000	1.9438
2.0675	11.72	1392000	1.9367
2.0675	11.79	1400000	1.9438
2.0731	11.86	1408000	1.9327
2.0731	11.92	1416000	1.9341
2.0774	11.99	1424000	1.9390
2.0774	12.06	1432000	1.9457
2.0738	12.13	1440000	1.9437
2.0738	12.19	1448000	1.9353
2.0667	12.26	1456000	1.9424
2.0667	12.33	1464000	1.9435
2.0674	12.4	1472000	1.9336
2.0674	12.46	1480000	1.9461
2.0704	12.53	1488000	1.9458
2.0704	12.6	1496000	1.9397
2.0691	12.67	1504000	1.9438
2.0691	12.73	1512000	1.9325
2.0727	12.8	1520000	1.9359
2.0727	12.87	1528000	1.9427
2.0715	12.93	1536000	1.9491
2.0715	13.0	1544000	1.9351
2.0692	13.07	1552000	1.9246
2.0692	13.14	1560000	1.9457
2.0711	13.2	1568000	1.9406
2.0711	13.27	1576000	1.9458
2.0735	13.34	1584000	1.9356
2.0735	13.41	1592000	1.9443
2.0707	13.47	1600000	1.9309
2.0707	13.54	1608000	1.9367
2.0776	13.61	1616000	1.9390
2.0776	13.68	1624000	1.9391
2.074	13.74	1632000	1.9459
2.074	13.81	1640000	1.9316
2.0681	13.88	1648000	1.9355
2.0681	13.95	1656000	1.9428
2.0671	14.01	1664000	1.9286
2.0671	14.08	1672000	1.9374
2.0672	14.15	1680000	1.9413
2.0672	14.21	1688000	1.9372
2.0675	14.28	1696000	1.9317
2.0675	14.35	1704000	1.9432
2.0665	14.42	1712000	1.9444
2.0665	14.48	1720000	1.9393
2.0645	14.55	1728000	1.9462
2.0645	14.62	1736000	1.9374
2.0712	14.69	1744000	1.9367
2.0712	14.75	1752000	1.9407
2.0689	14.82	1760000	1.9361
2.0689	14.89	1768000	1.9395
2.0657	14.96	1776000	1.9389
2.0657	15.02	1784000	1.9396
2.0781	15.09	1792000	1.9406
2.0781	15.16	1800000	1.9366
2.0631	15.23	1808000	1.9357
2.0631	15.29	1816000	1.9456
2.0738	15.36	1824000	1.9325
2.0738	15.43	1832000	1.9377
2.0726	15.49	1840000	1.9405
2.0726	15.56	1848000	1.9333
2.0699	15.63	1856000	1.9369
2.0699	15.7	1864000	1.9418
2.0764	15.76	1872000	1.9363
2.0764	15.83	1880000	1.9375
2.0779	15.9	1888000	1.9335
2.0779	15.97	1896000	1.9455
2.0693	16.03	1904000	1.9447
2.0693	16.1	1912000	1.9349
2.0716	16.17	1920000	1.9339
2.0716	16.24	1928000	1.9426
2.075	16.3	1936000	1.9439
2.075	16.37	1944000	1.9334
2.0751	16.44	1952000	1.9466
2.0751	16.51	1960000	1.9397
2.0734	16.57	1968000	1.9367
2.0734	16.64	1976000	1.9349
2.0685	16.71	1984000	1.9510
2.0685	16.77	1992000	1.9428
2.0706	16.84	2000000	1.9509
2.0706	16.91	2008000	1.9403
2.0716	16.98	2016000	1.9384
2.0716	17.04	2024000	1.9355
2.0741	17.11	2032000	1.9308
2.0741	17.18	2040000	1.9395
2.0714	17.25	2048000	1.9502
2.0714	17.31	2056000	1.9337
2.0696	17.38	2064000	1.9383
2.0696	17.45	2072000	1.9451
2.0729	17.52	2080000	1.9373
2.0729	17.58	2088000	1.9366
2.0716	17.65	2096000	1.9334
2.0716	17.72	2104000	1.9417
2.074	17.79	2112000	1.9408
2.074	17.85	2120000	1.9258
2.0745	17.92	2128000	1.9385
2.0745	17.99	2136000	1.9409
2.074	18.05	2144000	1.9342
2.074	18.12	2152000	1.9437
2.0666	18.19	2160000	1.9406
2.0666	18.26	2168000	1.9382
2.0657	18.32	2176000	1.9398
2.0657	18.39	2184000	1.9247
2.0692	18.46	2192000	1.9377
2.0692	18.53	2200000	1.9423
2.0726	18.59	2208000	1.9395
2.0726	18.66	2216000	1.9286
2.0688	18.73	2224000	1.9357
2.0688	18.8	2232000	1.9267
2.0732	18.86	2240000	1.9293
2.0732	18.93	2248000	1.9415
2.0697	19.0	2256000	1.9456
2.0697	19.07	2264000	1.9331
2.0747	19.13	2272000	1.9439
2.0747	19.2	2280000	1.9294
2.072	19.27	2288000	1.9305
2.072	19.33	2296000	1.9401
2.0609	19.4	2304000	1.9362
2.0609	19.47	2312000	1.9451
2.073	19.54	2320000	1.9352
2.073	19.6	2328000	1.9380
2.0793	19.67	2336000	1.9392
2.0793	19.74	2344000	1.9438
2.0787	19.81	2352000	1.9403
2.0787	19.87	2360000	1.9380
2.0694	19.94	2368000	1.9275
2.0694	20.01	2376000	1.9344
2.0649	20.08	2384000	1.9443
2.0649	20.14	2392000	1.9401
2.0727	20.21	2400000	1.9447

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0