2020-Q4-full_tweets

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9720

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.02	8000	2.2726
2.454	0.03	16000	2.1965
2.454	0.05	24000	2.1550
2.2713	0.07	32000	2.1327
2.2713	0.08	40000	2.1084
2.2285	0.1	48000	2.0920
2.2285	0.12	56000	2.0790
2.2116	0.13	64000	2.0766
2.2116	0.15	72000	2.0627
2.1857	0.17	80000	2.0600
2.1857	0.19	88000	2.0541
2.1716	0.2	96000	2.0404
2.1716	0.22	104000	2.0438
2.1594	0.24	112000	2.0344
2.1594	0.25	120000	2.0421
2.1584	0.27	128000	2.0309
2.1584	0.29	136000	2.0293
2.1426	0.3	144000	2.0262
2.1426	0.32	152000	2.0243
2.1494	0.34	160000	2.0235
2.1494	0.35	168000	2.0238
2.1466	0.37	176000	2.0158
2.1466	0.39	184000	2.0198
2.1389	0.4	192000	2.0098
2.1389	0.42	200000	2.0161
2.1312	0.44	208000	2.0185
2.1312	0.45	216000	2.0058
2.1404	0.47	224000	2.0143
2.1404	0.49	232000	2.0040
2.1385	0.51	240000	2.0060
2.1385	0.52	248000	2.0096
2.1356	0.54	256000	2.0073
2.1356	0.56	264000	2.0079
2.1297	0.57	272000	2.0068
2.1297	0.59	280000	2.0082
2.1319	0.61	288000	2.0070
2.1319	0.62	296000	2.0041
2.1296	0.64	304000	2.0038
2.1296	0.66	312000	2.0013
2.1289	0.67	320000	2.0043
2.1289	0.69	328000	2.0036
2.127	0.71	336000	2.0021
2.127	0.72	344000	2.0051
2.1244	0.74	352000	2.0006
2.1244	0.76	360000	2.0008
2.1271	0.77	368000	2.0028
2.1271	0.79	376000	2.0010
2.1258	0.81	384000	2.0008
2.1258	0.83	392000	1.9967
2.121	0.84	400000	2.0009
2.121	0.86	408000	1.9976
2.1288	0.88	416000	1.9993
2.1288	0.89	424000	1.9968
2.1358	0.91	432000	1.9999
2.1358	0.93	440000	1.9947
2.1339	0.94	448000	2.0011
2.1339	0.96	456000	2.0030
2.1256	0.98	464000	1.9871
2.1256	0.99	472000	1.9928
2.1304	1.01	480000	1.9876
2.1304	1.03	488000	1.9956
2.1224	1.04	496000	1.9979
2.1224	1.06	504000	1.9990
2.1274	1.08	512000	1.9970
2.1274	1.09	520000	1.9944
2.1215	1.11	528000	1.9924
2.1215	1.13	536000	1.9945
2.1246	1.15	544000	1.9916
2.1246	1.16	552000	1.9928
2.1305	1.18	560000	1.9927
2.1305	1.2	568000	1.9953
2.1204	1.21	576000	1.9892
2.1204	1.23	584000	1.9910
2.1171	1.25	592000	1.9920
2.1171	1.26	600000	1.9933
2.121	1.28	608000	1.9892
2.121	1.3	616000	1.9887
2.1238	1.31	624000	1.9917
2.1238	1.33	632000	1.9871
2.1235	1.35	640000	1.9852
2.1235	1.36	648000	1.9862
2.1266	1.38	656000	1.9866
2.1266	1.4	664000	1.9921
2.1236	1.41	672000	1.9807
2.1236	1.43	680000	1.9859
2.1278	1.45	688000	1.9925
2.1278	1.47	696000	1.9856
2.1116	1.48	704000	1.9882
2.1116	1.5	712000	1.9869
2.1128	1.52	720000	1.9819
2.1128	1.53	728000	1.9836
2.1208	1.55	736000	1.9819
2.1208	1.57	744000	1.9867
2.1248	1.58	752000	1.9893
2.1248	1.6	760000	1.9867
2.1181	1.62	768000	1.9826
2.1181	1.63	776000	1.9860
2.117	1.65	784000	1.9858
2.117	1.67	792000	1.9828
2.1203	1.68	800000	1.9846
2.1203	1.7	808000	1.9876
2.1219	1.72	816000	1.9816
2.1219	1.73	824000	1.9856
2.1226	1.75	832000	1.9833
2.1226	1.77	840000	1.9829
2.1218	1.79	848000	1.9870
2.1218	1.8	856000	1.9794
2.1207	1.82	864000	1.9860
2.1207	1.84	872000	1.9841
2.1173	1.85	880000	1.9851
2.1173	1.87	888000	1.9808
2.118	1.89	896000	1.9755
2.118	1.9	904000	1.9814
2.1085	1.92	912000	1.9834
2.1085	1.94	920000	1.9811
2.1213	1.95	928000	1.9837
2.1213	1.97	936000	1.9880
2.1254	1.99	944000	1.9802
2.1254	2.0	952000	1.9771
2.119	2.02	960000	1.9837
2.119	2.04	968000	1.9815
2.1217	2.05	976000	1.9791
2.1217	2.07	984000	1.9858
2.1196	2.09	992000	1.9823
2.1196	2.11	1000000	1.9849
2.1175	2.12	1008000	1.9832
2.1175	2.14	1016000	1.9795
2.1165	2.16	1024000	1.9848
2.1165	2.17	1032000	1.9813
2.1223	2.19	1040000	1.9791
2.1223	2.21	1048000	1.9791
2.1196	2.22	1056000	1.9724
2.1196	2.24	1064000	1.9779
2.1097	2.26	1072000	1.9785
2.1097	2.27	1080000	1.9842
2.109	2.29	1088000	1.9792
2.109	2.31	1096000	1.9804
2.1175	2.32	1104000	1.9811
2.1175	2.34	1112000	1.9813
2.1239	2.36	1120000	1.9742
2.1239	2.37	1128000	1.9759
2.1141	2.39	1136000	1.9835
2.1141	2.41	1144000	1.9814
2.1121	2.43	1152000	1.9753
2.1121	2.44	1160000	1.9796
2.1298	2.46	1168000	1.9720
2.1298	2.48	1176000	1.9822
2.1113	2.49	1184000	1.9772
2.1113	2.51	1192000	1.9779
2.1224	2.53	1200000	1.9760
2.1224	2.54	1208000	1.9823
2.1181	2.56	1216000	1.9836
2.1181	2.58	1224000	1.9754
2.1152	2.59	1232000	1.9764
2.1152	2.61	1240000	1.9771
2.1219	2.63	1248000	1.9774
2.1219	2.64	1256000	1.9790
2.115	2.66	1264000	1.9783
2.115	2.68	1272000	1.9829
2.1241	2.69	1280000	1.9844
2.1241	2.71	1288000	1.9781
2.1157	2.73	1296000	1.9808
2.1157	2.75	1304000	1.9820
2.1223	2.76	1312000	1.9812
2.1223	2.78	1320000	1.9811
2.1178	2.8	1328000	1.9779
2.1178	2.81	1336000	1.9761
2.1204	2.83	1344000	1.9772
2.1204	2.85	1352000	1.9724
2.1205	2.86	1360000	1.9777
2.1205	2.88	1368000	1.9721
2.1178	2.9	1376000	1.9768
2.1178	2.91	1384000	1.9802
2.1205	2.93	1392000	1.9759
2.1205	2.95	1400000	1.9817
2.1193	2.96	1408000	1.9788
2.1193	2.98	1416000	1.9770
2.1195	3.0	1424000	1.9769
2.1195	3.01	1432000	1.9848
2.1137	3.03	1440000	1.9747
2.1137	3.05	1448000	1.9745
2.12	3.07	1456000	1.9765
2.12	3.08	1464000	1.9776
2.123	3.1	1472000	1.9799
2.123	3.12	1480000	1.9737
2.1213	3.13	1488000	1.9775
2.1213	3.15	1496000	1.9783
2.1267	3.17	1504000	1.9806
2.1267	3.18	1512000	1.9764
2.1186	3.2	1520000	1.9695
2.1186	3.22	1528000	1.9783
2.1189	3.23	1536000	1.9774
2.1189	3.25	1544000	1.9781
2.1249	3.27	1552000	1.9740
2.1249	3.28	1560000	1.9787
2.1124	3.3	1568000	1.9799
2.1124	3.32	1576000	1.9734
2.1166	3.33	1584000	1.9763
2.1166	3.35	1592000	1.9798
2.1224	3.37	1600000	1.9741
2.1224	3.39	1608000	1.9781
2.1178	3.4	1616000	1.9705
2.1178	3.42	1624000	1.9754
2.1096	3.44	1632000	1.9738
2.1096	3.45	1640000	1.9785
2.1157	3.47	1648000	1.9745
2.1157	3.49	1656000	1.9788
2.1184	3.5	1664000	1.9739
2.1184	3.52	1672000	1.9722
2.1288	3.54	1680000	1.9729
2.1288	3.55	1688000	1.9782
2.1247	3.57	1696000	1.9772
2.1247	3.59	1704000	1.9759
2.1113	3.6	1712000	1.9696
2.1113	3.62	1720000	1.9751
2.124	3.64	1728000	1.9741
2.124	3.65	1736000	1.9780
2.1242	3.67	1744000	1.9777
2.1242	3.69	1752000	1.9724
2.1263	3.71	1760000	1.9775
2.1263	3.72	1768000	1.9779
2.1214	3.74	1776000	1.9786
2.1214	3.76	1784000	1.9770
2.1209	3.77	1792000	1.9809
2.1209	3.79	1800000	1.9754
2.1254	3.81	1808000	1.9769
2.1254	3.82	1816000	1.9782
2.1225	3.84	1824000	1.9799
2.1225	3.86	1832000	1.9781
2.1232	3.87	1840000	1.9752
2.1232	3.89	1848000	1.9749
2.1225	3.91	1856000	1.9787
2.1225	3.92	1864000	1.9765
2.118	3.94	1872000	1.9764
2.118	3.96	1880000	1.9767
2.1158	3.97	1888000	1.9775
2.1158	3.99	1896000	1.9775
2.1257	4.01	1904000	1.9750
2.1257	4.03	1912000	1.9756
2.122	4.04	1920000	1.9812
2.122	4.06	1928000	1.9753
2.1223	4.08	1936000	1.9788
2.1223	4.09	1944000	1.9773
2.1189	4.11	1952000	1.9798
2.1189	4.13	1960000	1.9724
2.1182	4.14	1968000	1.9813
2.1182	4.16	1976000	1.9821
2.118	4.18	1984000	1.9766
2.118	4.19	1992000	1.9779
2.1188	4.21	2000000	1.9700
2.1188	4.23	2008000	1.9783
2.1207	4.24	2016000	1.9744
2.1207	4.26	2024000	1.9800
2.1181	4.28	2032000	1.9769
2.1181	4.29	2040000	1.9770
2.1219	4.31	2048000	1.9745
2.1219	4.33	2056000	1.9719
2.1264	4.35	2064000	1.9766
2.1264	4.36	2072000	1.9753
2.1188	4.38	2080000	1.9752
2.1188	4.4	2088000	1.9787
2.1132	4.41	2096000	1.9755
2.1132	4.43	2104000	1.9824
2.1284	4.45	2112000	1.9788
2.1284	4.46	2120000	1.9768
2.1197	4.48	2128000	1.9800
2.1197	4.5	2136000	1.9771
2.1208	4.51	2144000	1.9769
2.1208	4.53	2152000	1.9770
2.1174	4.55	2160000	1.9727
2.1174	4.56	2168000	1.9772
2.1222	4.58	2176000	1.9709
2.1222	4.6	2184000	1.9768
2.1306	4.61	2192000	1.9721
2.1306	4.63	2200000	1.9730
2.1224	4.65	2208000	1.9756
2.1224	4.67	2216000	1.9703
2.1317	4.68	2224000	1.9788
2.1317	4.7	2232000	1.9760
2.1215	4.72	2240000	1.9795
2.1215	4.73	2248000	1.9747
2.1093	4.75	2256000	1.9798
2.1093	4.77	2264000	1.9734
2.1168	4.78	2272000	1.9769
2.1168	4.8	2280000	1.9767
2.1209	4.82	2288000	1.9758
2.1209	4.83	2296000	1.9794
2.1295	4.85	2304000	1.9806
2.1295	4.87	2312000	1.9778
2.1095	4.88	2320000	1.9740
2.1095	4.9	2328000	1.9753
2.1141	4.92	2336000	1.9768
2.1141	4.93	2344000	1.9744
2.1208	4.95	2352000	1.9785
2.1208	4.97	2360000	1.9829
2.1257	4.99	2368000	1.9744
2.1257	5.0	2376000	1.9829
2.1202	5.02	2384000	1.9729
2.1202	5.04	2392000	1.9804
2.1221	5.05	2400000	1.9803

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0