2020-Q3-25p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2870

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.02	8000	2.5705
2.7559	0.05	16000	2.4932
2.7559	0.07	24000	2.4516
2.5786	0.09	32000	2.4174
2.5786	0.11	40000	2.4071
2.5316	0.14	48000	2.3903
2.5316	0.16	56000	2.3744
2.5006	0.18	64000	2.3650
2.5006	0.2	72000	2.3600
2.483	0.23	80000	2.3548
2.483	0.25	88000	2.3485
2.4703	0.27	96000	2.3475
2.4703	0.29	104000	2.3384
2.47	0.32	112000	2.3330
2.47	0.34	120000	2.3354
2.4601	0.36	128000	2.3343
2.4601	0.38	136000	2.3282
2.4486	0.41	144000	2.3316
2.4486	0.43	152000	2.3180
2.4536	0.45	160000	2.3257
2.4536	0.47	168000	2.3222
2.4523	0.5	176000	2.3208
2.4523	0.52	184000	2.3218
2.4489	0.54	192000	2.3184
2.4489	0.56	200000	2.3225
2.4448	0.59	208000	2.3185
2.4448	0.61	216000	2.3139
2.4412	0.63	224000	2.3235
2.4412	0.65	232000	2.3148
2.442	0.68	240000	2.3146
2.442	0.7	248000	2.3145
2.4408	0.72	256000	2.3083
2.4408	0.74	264000	2.3068
2.4336	0.77	272000	2.3104
2.4336	0.79	280000	2.3147
2.4394	0.81	288000	2.3105
2.4394	0.83	296000	2.3135
2.4363	0.86	304000	2.3057
2.4363	0.88	312000	2.3050
2.4403	0.9	320000	2.3066
2.4403	0.92	328000	2.3076
2.4409	0.95	336000	2.3026
2.4409	0.97	344000	2.3045
2.4434	0.99	352000	2.3047
2.4434	1.01	360000	2.3080
2.4372	1.04	368000	2.3143
2.4372	1.06	376000	2.3049
2.4329	1.08	384000	2.3066
2.4329	1.1	392000	2.3050
2.437	1.13	400000	2.3012
2.437	1.15	408000	2.3033
2.4378	1.17	416000	2.3064
2.4378	1.19	424000	2.2984
2.4386	1.22	432000	2.3057
2.4386	1.24	440000	2.3035
2.4411	1.26	448000	2.2969
2.4411	1.28	456000	2.2930
2.4466	1.31	464000	2.3005
2.4466	1.33	472000	2.2975
2.4451	1.35	480000	2.3042
2.4451	1.37	488000	2.3061
2.4399	1.4	496000	2.2987
2.4399	1.42	504000	2.2967
2.4397	1.44	512000	2.3010
2.4397	1.47	520000	2.3019
2.4483	1.49	528000	2.3009
2.4483	1.51	536000	2.3048
2.4436	1.53	544000	2.3029
2.4436	1.56	552000	2.3026
2.4407	1.58	560000	2.3027
2.4407	1.6	568000	2.3061
2.4364	1.62	576000	2.2972
2.4364	1.65	584000	2.2967
2.4406	1.67	592000	2.2965
2.4406	1.69	600000	2.2966
2.4393	1.71	608000	2.2982
2.4393	1.74	616000	2.2993
2.4352	1.76	624000	2.2916
2.4352	1.78	632000	2.2931
2.4366	1.8	640000	2.3016
2.4366	1.83	648000	2.2984
2.4361	1.85	656000	2.2877
2.4361	1.87	664000	2.2983
2.437	1.89	672000	2.3033
2.437	1.92	680000	2.2928
2.4488	1.94	688000	2.2953
2.4488	1.96	696000	2.2945
2.4459	1.98	704000	2.2961
2.4459	2.01	712000	2.2899
2.4334	2.03	720000	2.2964
2.4334	2.05	728000	2.2896
2.4343	2.07	736000	2.2954
2.4343	2.1	744000	2.3004
2.4345	2.12	752000	2.2892
2.4345	2.14	760000	2.2996
2.4386	2.16	768000	2.2886
2.4386	2.19	776000	2.2974
2.434	2.21	784000	2.2882
2.434	2.23	792000	2.2965
2.4379	2.25	800000	2.2899
2.4379	2.28	808000	2.2938
2.4356	2.3	816000	2.2997
2.4356	2.32	824000	2.2942
2.4399	2.34	832000	2.2916
2.4399	2.37	840000	2.2934
2.437	2.39	848000	2.2978
2.437	2.41	856000	2.2834
2.4311	2.43	864000	2.2872
2.4311	2.46	872000	2.2928
2.4453	2.48	880000	2.2888
2.4453	2.5	888000	2.2933
2.4434	2.52	896000	2.2911
2.4434	2.55	904000	2.2929
2.443	2.57	912000	2.2926
2.443	2.59	920000	2.2908
2.4361	2.61	928000	2.2914
2.4361	2.64	936000	2.2878
2.44	2.66	944000	2.2872
2.44	2.68	952000	2.2857
2.4447	2.7	960000	2.2932
2.4447	2.73	968000	2.2918
2.4362	2.75	976000	2.2875
2.4362	2.77	984000	2.2900
2.4457	2.8	992000	2.2913
2.4457	2.82	1000000	2.2871
2.4474	2.84	1008000	2.2875
2.4474	2.86	1016000	2.2902
2.444	2.89	1024000	2.2878
2.444	2.91	1032000	2.2856
2.4316	2.93	1040000	2.2908
2.4316	2.95	1048000	2.2889
2.4388	2.98	1056000	2.2922
2.4388	3.0	1064000	2.2867
2.442	3.02	1072000	2.2912
2.442	3.04	1080000	2.2891
2.4388	3.07	1088000	2.2855
2.4388	3.09	1096000	2.2949
2.4296	3.11	1104000	2.2853
2.4296	3.13	1112000	2.2854
2.4411	3.16	1120000	2.2902
2.4411	3.18	1128000	2.2902
2.4354	3.2	1136000	2.2873
2.4354	3.22	1144000	2.2931
2.4436	3.25	1152000	2.2906
2.4436	3.27	1160000	2.2945
2.4372	3.29	1168000	2.2899
2.4372	3.31	1176000	2.2869
2.4327	3.34	1184000	2.2891
2.4327	3.36	1192000	2.2933
2.4387	3.38	1200000	2.2849
2.4387	3.4	1208000	2.2934
2.4433	3.43	1216000	2.2876
2.4433	3.45	1224000	2.2860
2.4396	3.47	1232000	2.2898
2.4396	3.49	1240000	2.2830
2.4332	3.52	1248000	2.2855
2.4332	3.54	1256000	2.2925
2.4332	3.56	1264000	2.2832
2.4332	3.58	1272000	2.2851
2.4307	3.61	1280000	2.2912
2.4307	3.63	1288000	2.2924
2.4432	3.65	1296000	2.2916
2.4432	3.67	1304000	2.2892
2.4319	3.7	1312000	2.2908
2.4319	3.72	1320000	2.2898
2.4394	3.74	1328000	2.2860
2.4394	3.76	1336000	2.2879
2.4462	3.79	1344000	2.2865
2.4462	3.81	1352000	2.2844
2.4373	3.83	1360000	2.2933
2.4373	3.85	1368000	2.2877
2.4436	3.88	1376000	2.2937
2.4436	3.9	1384000	2.2902
2.4387	3.92	1392000	2.2870
2.4387	3.94	1400000	2.2823
2.4384	3.97	1408000	2.2899
2.4384	3.99	1416000	2.2865
2.4389	4.01	1424000	2.2856
2.4389	4.03	1432000	2.2911
2.4408	4.06	1440000	2.2906
2.4408	4.08	1448000	2.2860
2.4424	4.1	1456000	2.2816
2.4424	4.12	1464000	2.2850
2.4446	4.15	1472000	2.2936
2.4446	4.17	1480000	2.2829
2.4419	4.19	1488000	2.2871
2.4419	4.22	1496000	2.2892
2.4327	4.24	1504000	2.2822
2.4327	4.26	1512000	2.2900
2.4346	4.28	1520000	2.2906
2.4346	4.31	1528000	2.2837
2.4342	4.33	1536000	2.2846
2.4342	4.35	1544000	2.2863
2.4381	4.37	1552000	2.2940
2.4381	4.4	1560000	2.2900
2.4445	4.42	1568000	2.2887
2.4445	4.44	1576000	2.2901
2.4306	4.46	1584000	2.2832
2.4306	4.49	1592000	2.2862
2.4348	4.51	1600000	2.2877
2.4348	4.53	1608000	2.2834
2.4446	4.55	1616000	2.2892
2.4446	4.58	1624000	2.2800
2.444	4.6	1632000	2.2891
2.444	4.62	1640000	2.2839
2.4335	4.64	1648000	2.2787
2.4335	4.67	1656000	2.2856
2.4369	4.69	1664000	2.2889
2.4369	4.71	1672000	2.2900
2.4446	4.73	1680000	2.2891
2.4446	4.76	1688000	2.2835
2.4334	4.78	1696000	2.2841
2.4334	4.8	1704000	2.2895
2.4426	4.82	1712000	2.2832
2.4426	4.85	1720000	2.2870
2.4434	4.87	1728000	2.2819
2.4434	4.89	1736000	2.2896
2.4382	4.91	1744000	2.2869
2.4382	4.94	1752000	2.2844
2.4405	4.96	1760000	2.2820
2.4405	4.98	1768000	2.2922
2.4507	5.0	1776000	2.2808
2.4507	5.03	1784000	2.2868
2.4437	5.05	1792000	2.2815
2.4437	5.07	1800000	2.2889
2.4373	5.09	1808000	2.2797
2.4373	5.12	1816000	2.2882
2.4368	5.14	1824000	2.2879
2.4368	5.16	1832000	2.2829
2.4398	5.18	1840000	2.2867
2.4398	5.21	1848000	2.2829
2.4469	5.23	1856000	2.2846
2.4469	5.25	1864000	2.2839
2.4457	5.27	1872000	2.2880
2.4457	5.3	1880000	2.2849
2.4444	5.32	1888000	2.2838
2.4444	5.34	1896000	2.2800
2.437	5.36	1904000	2.2915
2.437	5.39	1912000	2.2813
2.4415	5.41	1920000	2.2893
2.4415	5.43	1928000	2.2848
2.4472	5.45	1936000	2.2920
2.4472	5.48	1944000	2.2759
2.4418	5.5	1952000	2.2837
2.4418	5.52	1960000	2.2860
2.4406	5.54	1968000	2.2825
2.4406	5.57	1976000	2.2794
2.4359	5.59	1984000	2.2773
2.4359	5.61	1992000	2.2876
2.4416	5.64	2000000	2.2793
2.4416	5.66	2008000	2.2814
2.4327	5.68	2016000	2.2865
2.4327	5.7	2024000	2.2903
2.4395	5.73	2032000	2.2850
2.4395	5.75	2040000	2.2835
2.4379	5.77	2048000	2.2837
2.4379	5.79	2056000	2.2833
2.4471	5.82	2064000	2.2857
2.4471	5.84	2072000	2.2863
2.4443	5.86	2080000	2.2882
2.4443	5.88	2088000	2.2849
2.4406	5.91	2096000	2.2885
2.4406	5.93	2104000	2.2852
2.4502	5.95	2112000	2.2898
2.4502	5.97	2120000	2.2924
2.4356	6.0	2128000	2.2886
2.4356	6.02	2136000	2.2883
2.4431	6.04	2144000	2.2935
2.4431	6.06	2152000	2.2918
2.4379	6.09	2160000	2.2824
2.4379	6.11	2168000	2.2850
2.4504	6.13	2176000	2.2842
2.4504	6.15	2184000	2.2891
2.4352	6.18	2192000	2.2834
2.4352	6.2	2200000	2.2877
2.4385	6.22	2208000	2.2836
2.4385	6.24	2216000	2.2923
2.4401	6.27	2224000	2.2884
2.4401	6.29	2232000	2.2876
2.4396	6.31	2240000	2.2955
2.4396	6.33	2248000	2.2843
2.4384	6.36	2256000	2.2884
2.4384	6.38	2264000	2.2903
2.4365	6.4	2272000	2.2850
2.4365	6.42	2280000	2.2877
2.4361	6.45	2288000	2.2887
2.4361	6.47	2296000	2.2872
2.4409	6.49	2304000	2.2851
2.4409	6.51	2312000	2.2847
2.4423	6.54	2320000	2.2845
2.4423	6.56	2328000	2.2849
2.4409	6.58	2336000	2.2865
2.4409	6.6	2344000	2.2856
2.4468	6.63	2352000	2.2842
2.4468	6.65	2360000	2.2870
2.4461	6.67	2368000	2.2858
2.4461	6.69	2376000	2.2852
2.4469	6.72	2384000	2.2871
2.4469	6.74	2392000	2.2895
2.4413	6.76	2400000	2.2823

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0