label-transfer

This model is a fine-tuned version of saattrupdan/verdict-classifier on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0452
• F1 Macro: 0.9872
• F1 Misinformation: 0.9918
• F1 Factual: 0.9979
• F1 Other: 0.9720
• Prec Macro: 0.9842
• Prec Misinformation: 0.9958
• Prec Factual: 0.9979
• Prec Other: 0.9588

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 32
• total_train_batch_size: 2048
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1423
• num_epochs: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Macro	F1 Misinformation	F1 Factual	F1 Other	Prec Macro	Prec Misinformation	Prec Factual	Prec Other
0.4236	0.9	5	0.4070	0.8866	0.9477	0.9658	0.7463	0.9306	0.9075	0.9766	0.9077
0.4175	1.9	10	0.4001	0.8872	0.9480	0.9658	0.7477	0.9308	0.9079	0.9766	0.9080
0.4115	2.9	15	0.3884	0.8896	0.9487	0.9668	0.7534	0.9323	0.9093	0.9787	0.9090
0.3932	3.9	20	0.3719	0.8943	0.9509	0.9668	0.7652	0.9343	0.9133	0.9787	0.9110
0.3785	4.9	25	0.3505	0.8973	0.9522	0.9668	0.7730	0.9353	0.9160	0.9787	0.9112
0.3653	5.9	30	0.3266	0.9009	0.9535	0.9683	0.7809	0.9369	0.9186	0.9818	0.9104
0.3337	6.9	35	0.3028	0.9143	0.9599	0.9694	0.8137	0.9425	0.9310	0.9818	0.9148
0.3181	7.9	40	0.2796	0.9181	0.9624	0.9673	0.8245	0.9431	0.9361	0.9807	0.9125
0.2976	8.9	45	0.2570	0.9199	0.9633	0.9673	0.8291	0.9434	0.9383	0.9807	0.9113
0.2845	9.9	50	0.2349	0.9242	0.9658	0.9668	0.8401	0.9453	0.9430	0.9797	0.9131
0.2649	10.9	55	0.2134	0.9270	0.9673	0.9668	0.8470	0.9451	0.9472	0.9797	0.9086
0.2399	11.9	60	0.1929	0.9330	0.9704	0.9668	0.8619	0.9467	0.9547	0.9797	0.9057
0.224	12.9	65	0.1735	0.9369	0.9724	0.9673	0.8710	0.9467	0.9608	0.9797	0.8996
0.1992	13.9	70	0.1564	0.9496	0.9783	0.9711	0.8995	0.9531	0.9744	0.9809	0.9039
0.1908	14.9	75	0.1427	0.9501	0.9784	0.9711	0.9006	0.9519	0.9765	0.9799	0.8993
0.1785	15.9	80	0.1309	0.9542	0.9790	0.9765	0.9072	0.9549	0.9782	0.9791	0.9076
0.1637	16.9	85	0.1215	0.9531	0.9791	0.9745	0.9056	0.9536	0.9784	0.9750	0.9073
0.151	17.9	90	0.1131	0.9540	0.9787	0.9771	0.9064	0.9549	0.9776	0.9771	0.9099
0.1395	18.9	95	0.1049	0.9555	0.9790	0.9787	0.9088	0.9558	0.9784	0.9772	0.9119
0.1285	19.9	100	0.0963	0.9600	0.9799	0.9833	0.9169	0.9602	0.9798	0.9843	0.9164
0.1228	20.9	105	0.0887	0.9654	0.9829	0.9844	0.9289	0.9639	0.9850	0.9854	0.9215
0.1163	21.9	110	0.0832	0.9672	0.9839	0.9849	0.9329	0.9655	0.9864	0.9864	0.9237
0.1045	22.9	115	0.0792	0.9690	0.9849	0.9849	0.9374	0.9666	0.9883	0.9864	0.9251
0.0975	23.9	120	0.0758	0.9701	0.9854	0.9854	0.9396	0.9682	0.9880	0.9864	0.9303
0.0957	24.9	125	0.0731	0.9710	0.9856	0.9864	0.9411	0.9691	0.9883	0.9885	0.9305
0.0911	25.9	130	0.0702	0.9743	0.9862	0.9901	0.9467	0.9722	0.9891	0.9896	0.9377
0.0884	26.9	135	0.0676	0.9759	0.9875	0.9901	0.9502	0.9728	0.9916	0.9886	0.9381
0.087	27.9	140	0.0652	0.9770	0.9878	0.9912	0.9521	0.9739	0.9919	0.9906	0.9392
0.0813	28.9	145	0.0631	0.9791	0.9880	0.9938	0.9555	0.9758	0.9925	0.9938	0.9412
0.0758	29.9	150	0.0612	0.9805	0.9887	0.9943	0.9584	0.9767	0.9938	0.9938	0.9424
0.0734	30.9	155	0.0598	0.9796	0.9882	0.9943	0.9564	0.9762	0.9927	0.9938	0.9422
0.0713	31.9	160	0.0586	0.9798	0.9883	0.9943	0.9569	0.9765	0.9927	0.9938	0.9430
0.0662	32.9	165	0.0568	0.9805	0.9887	0.9943	0.9584	0.9768	0.9936	0.9938	0.9432
0.063	33.9	170	0.0552	0.9813	0.9893	0.9943	0.9602	0.9778	0.9938	0.9938	0.9459
0.0623	34.9	175	0.0538	0.9819	0.9897	0.9943	0.9616	0.9785	0.9941	0.9938	0.9477
0.0601	35.9	180	0.0531	0.9828	0.9901	0.9948	0.9635	0.9793	0.9947	0.9938	0.9496
0.0549	36.9	185	0.0521	0.9826	0.9900	0.9948	0.9631	0.9790	0.9947	0.9938	0.9487
0.0539	37.9	190	0.0512	0.9824	0.9898	0.9948	0.9626	0.9789	0.9944	0.9938	0.9486
0.0525	38.9	195	0.0503	0.9827	0.9898	0.9953	0.9630	0.9792	0.9944	0.9938	0.9495
0.0494	39.9	200	0.0498	0.9831	0.9898	0.9958	0.9635	0.9796	0.9944	0.9948	0.9496
0.0502	40.9	205	0.0489	0.9838	0.9901	0.9964	0.9650	0.9804	0.9947	0.9958	0.9506
0.0499	41.9	210	0.0483	0.9845	0.9904	0.9969	0.9663	0.9813	0.9947	0.9958	0.9532
0.0484	42.9	215	0.0480	0.9847	0.9905	0.9969	0.9668	0.9814	0.9950	0.9958	0.9533
0.0465	43.9	220	0.0477	0.9852	0.9908	0.9969	0.9678	0.9816	0.9955	0.9958	0.9534
0.0453	44.9	225	0.0474	0.9856	0.9911	0.9969	0.9687	0.9822	0.9955	0.9958	0.9551
0.0452	45.9	230	0.0471	0.9856	0.9911	0.9969	0.9687	0.9822	0.9955	0.9958	0.9551
0.0453	46.9	235	0.0469	0.9854	0.9910	0.9969	0.9682	0.9821	0.9953	0.9958	0.9551
0.043	47.9	240	0.0468	0.9858	0.9912	0.9969	0.9692	0.9825	0.9955	0.9958	0.9560
0.0428	48.9	245	0.0465	0.9856	0.9911	0.9969	0.9687	0.9824	0.9953	0.9958	0.9560
0.0414	49.9	250	0.0465	0.9852	0.9911	0.9964	0.9682	0.9820	0.9953	0.9948	0.9560
0.0388	50.9	255	0.0462	0.9852	0.9911	0.9964	0.9682	0.9820	0.9953	0.9948	0.9560
0.0404	51.9	260	0.0458	0.9852	0.9911	0.9964	0.9682	0.9820	0.9953	0.9948	0.9560
0.0382	52.9	265	0.0454	0.9856	0.9911	0.9969	0.9687	0.9824	0.9953	0.9958	0.9560
0.042	53.9	270	0.0443	0.9862	0.9911	0.9979	0.9697	0.9831	0.9953	0.9979	0.9561
0.0369	54.9	275	0.0438	0.9862	0.9911	0.9979	0.9697	0.9831	0.9953	0.9979	0.9561
0.0383	55.9	280	0.0437	0.9862	0.9911	0.9979	0.9697	0.9831	0.9953	0.9979	0.9561
0.0373	56.9	285	0.0438	0.9862	0.9911	0.9979	0.9696	0.9833	0.9950	0.9979	0.9569
0.0402	57.9	290	0.0440	0.9862	0.9911	0.9979	0.9696	0.9833	0.9950	0.9979	0.9569
0.0389	58.9	295	0.0443	0.9858	0.9908	0.9979	0.9687	0.9831	0.9944	0.9979	0.9568
0.0361	59.9	300	0.0443	0.9860	0.9910	0.9979	0.9692	0.9832	0.9947	0.9979	0.9569
0.0369	60.9	305	0.0442	0.9860	0.9910	0.9979	0.9692	0.9832	0.9947	0.9979	0.9569
0.0353	61.9	310	0.0442	0.9862	0.9911	0.9979	0.9696	0.9833	0.9950	0.9979	0.9569
0.035	62.9	315	0.0446	0.9860	0.9910	0.9979	0.9692	0.9832	0.9947	0.9979	0.9569
0.0352	63.9	320	0.0449	0.9864	0.9912	0.9979	0.9701	0.9834	0.9953	0.9979	0.9570
0.0336	64.9	325	0.0451	0.9860	0.9910	0.9979	0.9692	0.9832	0.9947	0.9979	0.9569
0.0317	65.9	330	0.0448	0.9860	0.9910	0.9979	0.9692	0.9832	0.9947	0.9979	0.9569
0.0334	66.9	335	0.0447	0.9866	0.9914	0.9979	0.9705	0.9843	0.9944	0.9979	0.9605
0.0316	67.9	340	0.0447	0.9860	0.9910	0.9979	0.9691	0.9834	0.9944	0.9979	0.9577
0.0329	68.9	345	0.0451	0.9866	0.9914	0.9979	0.9706	0.9835	0.9955	0.9979	0.9570
0.0326	69.9	350	0.0454	0.9866	0.9914	0.9979	0.9706	0.9835	0.9955	0.9979	0.9570
0.032	70.9	355	0.0453	0.9868	0.9915	0.9979	0.9711	0.9838	0.9955	0.9979	0.9579
0.0325	71.9	360	0.0450	0.9864	0.9912	0.9979	0.9701	0.9836	0.9950	0.9979	0.9578
0.0319	72.9	365	0.0446	0.9868	0.9915	0.9979	0.9711	0.9838	0.9955	0.9979	0.9579
0.0326	73.9	370	0.0444	0.9868	0.9915	0.9979	0.9711	0.9838	0.9955	0.9979	0.9579
0.0315	74.9	375	0.0442	0.9873	0.9918	0.9979	0.9721	0.9840	0.9961	0.9979	0.9580
0.0304	75.9	380	0.0442	0.9866	0.9914	0.9979	0.9706	0.9837	0.9953	0.9979	0.9579
0.03	76.9	385	0.0444	0.9864	0.9912	0.9979	0.9702	0.9832	0.9955	0.9979	0.9561
0.0296	77.9	390	0.0448	0.9862	0.9911	0.9979	0.9697	0.9831	0.9953	0.9979	0.9561
0.0307	78.9	395	0.0452	0.9866	0.9914	0.9979	0.9706	0.9837	0.9953	0.9979	0.9579
0.0296	79.9	400	0.0453	0.9862	0.9911	0.9979	0.9697	0.9831	0.9953	0.9979	0.9561
0.0292	80.9	405	0.0454	0.9862	0.9911	0.9979	0.9697	0.9831	0.9953	0.9979	0.9561
0.0293	81.9	410	0.0452	0.9862	0.9911	0.9979	0.9697	0.9829	0.9955	0.9979	0.9552
0.0292	82.9	415	0.0454	0.9862	0.9911	0.9979	0.9697	0.9829	0.9955	0.9979	0.9552
0.0281	83.9	420	0.0454	0.9866	0.9914	0.9979	0.9706	0.9833	0.9958	0.9979	0.9562
0.0298	84.9	425	0.0452	0.9872	0.9918	0.9979	0.9720	0.9842	0.9958	0.9979	0.9588

Framework versions

• Transformers 4.23.1
• Pytorch 1.12.1+cu113
• Datasets 2.6.1
• Tokenizers 0.13.1