DouglasPontes/2020-Q1-25p-filtered-random

2020-Q1-25p-filtered

This model is a fine-tuned version of cardiffnlp/twitter-roberta-base-2019-90m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2233

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4.1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 2400000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.02	8000	2.5883
2.7561	0.04	16000	2.4958
2.7561	0.07	24000	2.4376
2.531	0.09	32000	2.4090
2.531	0.11	40000	2.3791
2.4627	0.13	48000	2.3666
2.4627	0.15	56000	2.3457
2.4252	0.17	64000	2.3380
2.4252	0.2	72000	2.3298
2.4061	0.22	80000	2.3253
2.4061	0.24	88000	2.3177
2.395	0.26	96000	2.3131
2.395	0.28	104000	2.3058
2.3843	0.31	112000	2.3010
2.3843	0.33	120000	2.2925
2.3738	0.35	128000	2.2916
2.3738	0.37	136000	2.2947
2.3686	0.39	144000	2.2835
2.3686	0.42	152000	2.2864
2.3615	0.44	160000	2.2834
2.3615	0.46	168000	2.2768
2.3515	0.48	176000	2.2803
2.3515	0.5	184000	2.2804
2.3508	0.52	192000	2.2754
2.3508	0.55	200000	2.2767
2.35	0.57	208000	2.2742
2.35	0.59	216000	2.2722
2.3385	0.61	224000	2.2661
2.3385	0.63	232000	2.2706
2.3393	0.66	240000	2.2633
2.3393	0.68	248000	2.2648
2.3392	0.7	256000	2.2656
2.3392	0.72	264000	2.2660
2.3336	0.74	272000	2.2657
2.3336	0.76	280000	2.2605
2.3324	0.79	288000	2.2615
2.3324	0.81	296000	2.2551
2.3312	0.83	304000	2.2581
2.3312	0.85	312000	2.2626
2.3352	0.87	320000	2.2576
2.3352	0.9	328000	2.2553
2.3287	0.92	336000	2.2591
2.3287	0.94	344000	2.2558
2.321	0.96	352000	2.2603
2.321	0.98	360000	2.2569
2.3278	1.01	368000	2.2544
2.3278	1.03	376000	2.2604
2.319	1.05	384000	2.2535
2.319	1.07	392000	2.2420
2.3151	1.09	400000	2.2583
2.3151	1.11	408000	2.2535
2.3144	1.14	416000	2.2582
2.3144	1.16	424000	2.2496
2.3191	1.18	432000	2.2532
2.3191	1.2	440000	2.2515
2.3168	1.22	448000	2.2501
2.3168	1.25	456000	2.2453
2.3156	1.27	464000	2.2445
2.3156	1.29	472000	2.2485
2.3178	1.31	480000	2.2493
2.3178	1.33	488000	2.2443
2.3113	1.35	496000	2.2493
2.3113	1.38	504000	2.2493
2.3116	1.4	512000	2.2483
2.3116	1.42	520000	2.2459
2.3166	1.44	528000	2.2481
2.3166	1.46	536000	2.2542
2.3158	1.49	544000	2.2443
2.3158	1.51	552000	2.2402
2.3148	1.53	560000	2.2449
2.3148	1.55	568000	2.2415
2.3145	1.57	576000	2.2471
2.3145	1.6	584000	2.2469
2.3119	1.62	592000	2.2445
2.3119	1.64	600000	2.2487
2.3045	1.66	608000	2.2456
2.3045	1.68	616000	2.2466
2.3046	1.7	624000	2.2357
2.3046	1.73	632000	2.2448
2.3083	1.75	640000	2.2381
2.3083	1.77	648000	2.2439
2.3065	1.79	656000	2.2402
2.3065	1.81	664000	2.2439
2.307	1.84	672000	2.2409
2.307	1.86	680000	2.2426
2.3026	1.88	688000	2.2387
2.3026	1.9	696000	2.2357
2.2949	1.92	704000	2.2379
2.2949	1.95	712000	2.2408
2.2951	1.97	720000	2.2432
2.2951	1.99	728000	2.2444
2.3011	2.01	736000	2.2382
2.3011	2.03	744000	2.2391
2.3017	2.05	752000	2.2363
2.3017	2.08	760000	2.2444
2.2978	2.1	768000	2.2370
2.2978	2.12	776000	2.2350
2.2961	2.14	784000	2.2347
2.2961	2.16	792000	2.2386
2.2968	2.19	800000	2.2322
2.2968	2.21	808000	2.2403
2.2962	2.23	816000	2.2347
2.2962	2.25	824000	2.2398
2.2984	2.27	832000	2.2358
2.2984	2.29	840000	2.2412
2.3029	2.32	848000	2.2386
2.3029	2.34	856000	2.2346
2.2985	2.36	864000	2.2323
2.2985	2.38	872000	2.2387
2.2922	2.4	880000	2.2303
2.2922	2.43	888000	2.2326
2.2967	2.45	896000	2.2422
2.2967	2.47	904000	2.2350
2.2917	2.49	912000	2.2299
2.2917	2.51	920000	2.2308
2.2912	2.54	928000	2.2345
2.2912	2.56	936000	2.2264
2.2887	2.58	944000	2.2361
2.2887	2.6	952000	2.2319
2.2956	2.62	960000	2.2340
2.2956	2.64	968000	2.2356
2.2927	2.67	976000	2.2366
2.2927	2.69	984000	2.2335
2.2872	2.71	992000	2.2330
2.2872	2.73	1000000	2.2251
2.2936	2.75	1008000	2.2327
2.2936	2.78	1016000	2.2326
2.2899	2.8	1024000	2.2307
2.2899	2.82	1032000	2.2291
2.2931	2.84	1040000	2.2285
2.2931	2.86	1048000	2.2327
2.3042	2.88	1056000	2.2367
2.3042	2.91	1064000	2.2345
2.2864	2.93	1072000	2.2267
2.2864	2.95	1080000	2.2343
2.2933	2.97	1088000	2.2354
2.2933	2.99	1096000	2.2260
2.2909	3.02	1104000	2.2341
2.2909	3.04	1112000	2.2266
2.2889	3.06	1120000	2.2253
2.2889	3.08	1128000	2.2255
2.292	3.1	1136000	2.2194
2.292	3.13	1144000	2.2319
2.282	3.15	1152000	2.2221
2.282	3.17	1160000	2.2273
2.2827	3.19	1168000	2.2296
2.2827	3.21	1176000	2.2332
2.2937	3.23	1184000	2.2302
2.2937	3.26	1192000	2.2262
2.2845	3.28	1200000	2.2318
2.2845	3.3	1208000	2.2291
2.284	3.32	1216000	2.2327
2.284	3.34	1224000	2.2308
2.2923	3.37	1232000	2.2264
2.2923	3.39	1240000	2.2390
2.2859	3.41	1248000	2.2310
2.2859	3.43	1256000	2.2287
2.2879	3.45	1264000	2.2284
2.2879	3.47	1272000	2.2228
2.292	3.5	1280000	2.2296
2.292	3.52	1288000	2.2329
2.2827	3.54	1296000	2.2263
2.2827	3.56	1304000	2.2324
2.2829	3.58	1312000	2.2232
2.2829	3.61	1320000	2.2273
2.2863	3.63	1328000	2.2296
2.2863	3.65	1336000	2.2294
2.2796	3.67	1344000	2.2283
2.2796	3.69	1352000	2.2280
2.2835	3.72	1360000	2.2264
2.2835	3.74	1368000	2.2224
2.2875	3.76	1376000	2.2219
2.2875	3.78	1384000	2.2243
2.2792	3.8	1392000	2.2320
2.2792	3.82	1400000	2.2273
2.2932	3.85	1408000	2.2257
2.2932	3.87	1416000	2.2360
2.2899	3.89	1424000	2.2277
2.2899	3.91	1432000	2.2275
2.2859	3.93	1440000	2.2287
2.2859	3.96	1448000	2.2211
2.2876	3.98	1456000	2.2236
2.2876	4.0	1464000	2.2288
2.2879	4.02	1472000	2.2226
2.2879	4.04	1480000	2.2242
2.282	4.06	1488000	2.2286
2.282	4.09	1496000	2.2210
2.2828	4.11	1504000	2.2304
2.2828	4.13	1512000	2.2310
2.2765	4.15	1520000	2.2295
2.2765	4.17	1528000	2.2276
2.2839	4.2	1536000	2.2260
2.2839	4.22	1544000	2.2255
2.2845	4.24	1552000	2.2200
2.2845	4.26	1560000	2.2228
2.2816	4.28	1568000	2.2322
2.2816	4.31	1576000	2.2250
2.2965	4.33	1584000	2.2242
2.2965	4.35	1592000	2.2295
2.2806	4.37	1600000	2.2198
2.2806	4.39	1608000	2.2301
2.2868	4.41	1616000	2.2309
2.2868	4.44	1624000	2.2270
2.2907	4.46	1632000	2.2291
2.2907	4.48	1640000	2.2269
2.2809	4.5	1648000	2.2261
2.2809	4.52	1656000	2.2318
2.2876	4.55	1664000	2.2252
2.2876	4.57	1672000	2.2248
2.2844	4.59	1680000	2.2223
2.2844	4.61	1688000	2.2250
2.2841	4.63	1696000	2.2278
2.2841	4.65	1704000	2.2226
2.2851	4.68	1712000	2.2274
2.2851	4.7	1720000	2.2247
2.2863	4.72	1728000	2.2239
2.2863	4.74	1736000	2.2227
2.2788	4.76	1744000	2.2234
2.2788	4.79	1752000	2.2293
2.2849	4.81	1760000	2.2199
2.2849	4.83	1768000	2.2309
2.2826	4.85	1776000	2.2235
2.2826	4.87	1784000	2.2292
2.2809	4.9	1792000	2.2248
2.2809	4.92	1800000	2.2187
2.2865	4.94	1808000	2.2331
2.2865	4.96	1816000	2.2244
2.2773	4.98	1824000	2.2246
2.2773	5.0	1832000	2.2315
2.2738	5.03	1840000	2.2319
2.2738	5.05	1848000	2.2258
2.2806	5.07	1856000	2.2241
2.2806	5.09	1864000	2.2228
2.2822	5.11	1872000	2.2218
2.2822	5.14	1880000	2.2276
2.2866	5.16	1888000	2.2233
2.2866	5.18	1896000	2.2266
2.2831	5.2	1904000	2.2231
2.2831	5.22	1912000	2.2241
2.2875	5.24	1920000	2.2263
2.2875	5.27	1928000	2.2234
2.2802	5.29	1936000	2.2269
2.2802	5.31	1944000	2.2253
2.2905	5.33	1952000	2.2191
2.2905	5.35	1960000	2.2217
2.282	5.38	1968000	2.2212
2.282	5.4	1976000	2.2213
2.2798	5.42	1984000	2.2218
2.2798	5.44	1992000	2.2222
2.2864	5.46	2000000	2.2212
2.2864	5.49	2008000	2.2282
2.2867	5.51	2016000	2.2304
2.2867	5.53	2024000	2.2222
2.2834	5.55	2032000	2.2285
2.2834	5.57	2040000	2.2230
2.2851	5.59	2048000	2.2237
2.2851	5.62	2056000	2.2283
2.2774	5.64	2064000	2.2232
2.2774	5.66	2072000	2.2282
2.277	5.68	2080000	2.2271
2.277	5.7	2088000	2.2256
2.2868	5.73	2096000	2.2252
2.2868	5.75	2104000	2.2285
2.2727	5.77	2112000	2.2251
2.2727	5.79	2120000	2.2239
2.2803	5.81	2128000	2.2287
2.2803	5.84	2136000	2.2274
2.2785	5.86	2144000	2.2227
2.2785	5.88	2152000	2.2267
2.2829	5.9	2160000	2.2251
2.2829	5.92	2168000	2.2228
2.2816	5.94	2176000	2.2235
2.2816	5.97	2184000	2.2289
2.283	5.99	2192000	2.2238
2.283	6.01	2200000	2.2245
2.2761	6.03	2208000	2.2297
2.2761	6.05	2216000	2.2300
2.2823	6.08	2224000	2.2268
2.2823	6.1	2232000	2.2252
2.2715	6.12	2240000	2.2240
2.2715	6.14	2248000	2.2233
2.2809	6.16	2256000	2.2238
2.2809	6.18	2264000	2.2204
2.2823	6.21	2272000	2.2218
2.2823	6.23	2280000	2.2295
2.2848	6.25	2288000	2.2298
2.2848	6.27	2296000	2.2299
2.2847	6.29	2304000	2.2246
2.2847	6.32	2312000	2.2230
2.2783	6.34	2320000	2.2260
2.2783	6.36	2328000	2.2176
2.2791	6.38	2336000	2.2211
2.2791	6.4	2344000	2.2262
2.2797	6.43	2352000	2.2293
2.2797	6.45	2360000	2.2219
2.2784	6.47	2368000	2.2249
2.2784	6.49	2376000	2.2216
2.271	6.51	2384000	2.2256
2.271	6.53	2392000	2.2296
2.2787	6.56	2400000	2.2275

Framework versions

• Transformers 4.35.0.dev0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.0