2020-Q3-90p-filtered-random

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2598

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.17	8000	2.5349
2.7955	0.34	16000	2.4448
2.7955	0.51	24000	2.3949
2.5335	0.67	32000	2.3699
2.5335	0.84	40000	2.3544
2.4757	1.01	48000	2.3477
2.4757	1.18	56000	2.3281
2.446	1.35	64000	2.3171
2.446	1.52	72000	2.3082
2.4291	1.68	80000	2.3170
2.4291	1.85	88000	2.2962
2.4275	2.02	96000	2.3044
2.4275	2.19	104000	2.2738
2.4148	2.36	112000	2.2927
2.4148	2.53	120000	2.2684
2.4062	2.69	128000	2.2891
2.4062	2.86	136000	2.2789
2.4022	3.03	144000	2.2659
2.4022	3.2	152000	2.2824
2.3943	3.37	160000	2.2684
2.3943	3.54	168000	2.2683
2.3957	3.71	176000	2.2737
2.3957	3.87	184000	2.2779
2.3976	4.04	192000	2.2710
2.3976	4.21	200000	2.2495
2.3933	4.38	208000	2.2660
2.3933	4.55	216000	2.2687
2.4039	4.72	224000	2.2581
2.4039	4.88	232000	2.2656
2.3966	5.05	240000	2.2543
2.3966	5.22	248000	2.2768
2.3902	5.39	256000	2.2551
2.3902	5.56	264000	2.2782
2.3906	5.73	272000	2.2639
2.3906	5.89	280000	2.2585
2.3849	6.06	288000	2.2540
2.3849	6.23	296000	2.2749
2.3805	6.4	304000	2.2503
2.3805	6.57	312000	2.2739
2.3873	6.74	320000	2.2541
2.3873	6.91	328000	2.2512
2.3942	7.07	336000	2.2595
2.3942	7.24	344000	2.2603
2.386	7.41	352000	2.2575
2.386	7.58	360000	2.2789
2.3806	7.75	368000	2.2650
2.3806	7.92	376000	2.2706
2.3883	8.08	384000	2.2652
2.3883	8.25	392000	2.2540
2.3922	8.42	400000	2.2683
2.3922	8.59	408000	2.2638
2.3887	8.76	416000	2.2535
2.3887	8.93	424000	2.2529
2.3818	9.09	432000	2.2483
2.3818	9.26	440000	2.2574
2.387	9.43	448000	2.2624
2.387	9.6	456000	2.2664
2.3839	9.77	464000	2.2572
2.3839	9.94	472000	2.2524
2.3901	10.11	480000	2.2533
2.3901	10.27	488000	2.2501
2.382	10.44	496000	2.2669
2.382	10.61	504000	2.2596
2.3829	10.78	512000	2.2705
2.3829	10.95	520000	2.2553
2.3963	11.12	528000	2.2741
2.3963	11.28	536000	2.2664
2.3843	11.45	544000	2.2532
2.3843	11.62	552000	2.2720
2.3853	11.79	560000	2.2532
2.3853	11.96	568000	2.2700
2.3907	12.13	576000	2.2571
2.3907	12.29	584000	2.2523
2.3865	12.46	592000	2.2458
2.3865	12.63	600000	2.2647
2.3827	12.8	608000	2.2490
2.3827	12.97	616000	2.2624
2.3869	13.14	624000	2.2538
2.3869	13.3	632000	2.2357
2.3958	13.47	640000	2.2509
2.3958	13.64	648000	2.2690
2.3852	13.81	656000	2.2476
2.3852	13.98	664000	2.2721
2.3889	14.15	672000	2.2537
2.3889	14.32	680000	2.2723
2.3839	14.48	688000	2.2664
2.3839	14.65	696000	2.2726
2.3884	14.82	704000	2.2652
2.3884	14.99	712000	2.2633
2.3827	15.16	720000	2.2681
2.3827	15.33	728000	2.2543
2.3861	15.49	736000	2.2634
2.3861	15.66	744000	2.2707
2.3812	15.83	752000	2.2575
2.3812	16.0	760000	2.2549
2.3862	16.17	768000	2.2446
2.3862	16.34	776000	2.2617
2.3859	16.5	784000	2.2505
2.3859	16.67	792000	2.2720
2.3873	16.84	800000	2.2521
2.3873	17.01	808000	2.2543
2.381	17.18	816000	2.2675
2.381	17.35	824000	2.2545
2.3851	17.52	832000	2.2489
2.3851	17.68	840000	2.2606
2.3878	17.85	848000	2.2580
2.3878	18.02	856000	2.2604
2.3812	18.19	864000	2.2631
2.3812	18.36	872000	2.2505
2.3849	18.53	880000	2.2658
2.3849	18.69	888000	2.2567
2.3833	18.86	896000	2.2533
2.3833	19.03	904000	2.2456
2.3847	19.2	912000	2.2533
2.3847	19.37	920000	2.2575
2.3869	19.54	928000	2.2668
2.3869	19.7	936000	2.2599
2.3867	19.87	944000	2.2680
2.3867	20.04	952000	2.2669
2.3942	20.21	960000	2.2483
2.3942	20.38	968000	2.2734
2.3863	20.55	976000	2.2623
2.3863	20.72	984000	2.2650
2.3924	20.88	992000	2.2603
2.3924	21.05	1000000	2.2708
2.3871	21.22	1008000	2.2512
2.3871	21.39	1016000	2.2568
2.3827	21.56	1024000	2.2676
2.3827	21.73	1032000	2.2710
2.3799	21.89	1040000	2.2804
2.3799	22.06	1048000	2.2499
2.3863	22.23	1056000	2.2557
2.3863	22.4	1064000	2.2604
2.3858	22.57	1072000	2.2832
2.3858	22.74	1080000	2.2443
2.3859	22.9	1088000	2.2604
2.3859	23.07	1096000	2.2631
2.3846	23.24	1104000	2.2690
2.3846	23.41	1112000	2.2595
2.3887	23.58	1120000	2.2501
2.3887	23.75	1128000	2.2533
2.3856	23.92	1136000	2.2529
2.3856	24.08	1144000	2.2456
2.3856	24.25	1152000	2.2544
2.3856	24.42	1160000	2.2554
2.3867	24.59	1168000	2.2596
2.3867	24.76	1176000	2.2522
2.3795	24.93	1184000	2.2493
2.3795	25.09	1192000	2.2609
2.3926	25.26	1200000	2.2658
2.3926	25.43	1208000	2.2593
2.3887	25.6	1216000	2.2704
2.3887	25.77	1224000	2.2632
2.3926	25.94	1232000	2.2628
2.3926	26.1	1240000	2.2657
2.3809	26.27	1248000	2.2546
2.3809	26.44	1256000	2.2596
2.3878	26.61	1264000	2.2545
2.3878	26.78	1272000	2.2668
2.3861	26.95	1280000	2.2534
2.3861	27.12	1288000	2.2612
2.3815	27.28	1296000	2.2441
2.3815	27.45	1304000	2.2714
2.3861	27.62	1312000	2.2604
2.3861	27.79	1320000	2.2535
2.388	27.96	1328000	2.2466
2.388	28.13	1336000	2.2581
2.3864	28.29	1344000	2.2572
2.3864	28.46	1352000	2.2381
2.39	28.63	1360000	2.2398
2.39	28.8	1368000	2.2695
2.39	28.97	1376000	2.2628
2.39	29.14	1384000	2.2599
2.3804	29.3	1392000	2.2628
2.3804	29.47	1400000	2.2722
2.3858	29.64	1408000	2.2490
2.3858	29.81	1416000	2.2627
2.3804	29.98	1424000	2.2623
2.3804	30.15	1432000	2.2522
2.3834	30.32	1440000	2.2633
2.3834	30.48	1448000	2.2553
2.3853	30.65	1456000	2.2391
2.3853	30.82	1464000	2.2616
2.3946	30.99	1472000	2.2631
2.3946	31.16	1480000	2.2639
2.385	31.33	1488000	2.2736
2.385	31.49	1496000	2.2715
2.387	31.66	1504000	2.2557
2.387	31.83	1512000	2.2583
2.3831	32.0	1520000	2.2544
2.3831	32.17	1528000	2.2756
2.3835	32.34	1536000	2.2794
2.3835	32.5	1544000	2.2648
2.3857	32.67	1552000	2.2563
2.3857	32.84	1560000	2.2537
2.3856	33.01	1568000	2.2610
2.3856	33.18	1576000	2.2646
2.3902	33.35	1584000	2.2545
2.3902	33.52	1592000	2.2710
2.3897	33.68	1600000	2.2601
2.3897	33.85	1608000	2.2543
2.3866	34.02	1616000	2.2526
2.3866	34.19	1624000	2.2629
2.3823	34.36	1632000	2.2617
2.3823	34.53	1640000	2.2520
2.3874	34.69	1648000	2.2612
2.3874	34.86	1656000	2.2569
2.3895	35.03	1664000	2.2633
2.3895	35.2	1672000	2.2593
2.3857	35.37	1680000	2.2651
2.3857	35.54	1688000	2.2567
2.3811	35.7	1696000	2.2534
2.3811	35.87	1704000	2.2633
2.3944	36.04	1712000	2.2504
2.3944	36.21	1720000	2.2519
2.3883	36.38	1728000	2.2572
2.3883	36.55	1736000	2.2576
2.3859	36.72	1744000	2.2719
2.3859	36.88	1752000	2.2668
2.3914	37.05	1760000	2.2509
2.3914	37.22	1768000	2.2601
2.3848	37.39	1776000	2.2687
2.3848	37.56	1784000	2.2513
2.3903	37.73	1792000	2.2519
2.3903	37.89	1800000	2.2594
2.3822	38.06	1808000	2.2565
2.3822	38.23	1816000	2.2812
2.383	38.4	1824000	2.2589
2.383	38.57	1832000	2.2560
2.3868	38.74	1840000	2.2648
2.3868	38.9	1848000	2.2507
2.3775	39.07	1856000	2.2570
2.3775	39.24	1864000	2.2549
2.3818	39.41	1872000	2.2583
2.3818	39.58	1880000	2.2610
2.3887	39.75	1888000	2.2629
2.3887	39.91	1896000	2.2739
2.3893	40.08	1904000	2.2657
2.3893	40.25	1912000	2.2507
2.3826	40.42	1920000	2.2506
2.3826	40.59	1928000	2.2630
2.3842	40.76	1936000	2.2716
2.3842	40.93	1944000	2.2642
2.3866	41.09	1952000	2.2451
2.3866	41.26	1960000	2.2521
2.3857	41.43	1968000	2.2457
2.3857	41.6	1976000	2.2575
2.3943	41.77	1984000	2.2659
2.3943	41.94	1992000	2.2608
2.387	42.1	2000000	2.2687
2.387	42.27	2008000	2.2718
2.387	42.44	2016000	2.2629
2.387	42.61	2024000	2.2283
2.3804	42.78	2032000	2.2422
2.3804	42.95	2040000	2.2431
2.3842	43.11	2048000	2.2689
2.3842	43.28	2056000	2.2586
2.3856	43.45	2064000	2.2590
2.3856	43.62	2072000	2.2602
2.3843	43.79	2080000	2.2557
2.3843	43.96	2088000	2.2776
2.3891	44.13	2096000	2.2554
2.3891	44.29	2104000	2.2615
2.3811	44.46	2112000	2.2591
2.3811	44.63	2120000	2.2600
2.3874	44.8	2128000	2.2595
2.3874	44.97	2136000	2.2762
2.3822	45.14	2144000	2.2516
2.3822	45.3	2152000	2.2530
2.3933	45.47	2160000	2.2652
2.3933	45.64	2168000	2.2480
2.3853	45.81	2176000	2.2717
2.3853	45.98	2184000	2.2569
2.3917	46.15	2192000	2.2564
2.3917	46.31	2200000	2.2512
2.3859	46.48	2208000	2.2612
2.3859	46.65	2216000	2.2609
2.3879	46.82	2224000	2.2552
2.3879	46.99	2232000	2.2568
2.3823	47.16	2240000	2.2507
2.3823	47.33	2248000	2.2762
2.388	47.49	2256000	2.2522
2.388	47.66	2264000	2.2532
2.3773	47.83	2272000	2.2490
2.3773	48.0	2280000	2.2648
2.3828	48.17	2288000	2.2500
2.3828	48.34	2296000	2.2534
2.3816	48.5	2304000	2.2515
2.3816	48.67	2312000	2.2702
2.3784	48.84	2320000	2.2584
2.3784	49.01	2328000	2.2382
2.3863	49.18	2336000	2.2604
2.3863	49.35	2344000	2.2607
2.3863	49.51	2352000	2.2646
2.3863	49.68	2360000	2.2534
2.3873	49.85	2368000	2.2742
2.3873	50.02	2376000	2.2687
2.39	50.19	2384000	2.2581
2.39	50.36	2392000	2.2460
2.3937	50.53	2400000	2.2642

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0