arabert_cross_vocabulary_task2_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7744
• Qwk: 0.5129
• Mse: 0.7751

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1333	2	6.0135	0.0063	6.0118
No log	0.2667	4	3.5790	0.0426	3.5769
No log	0.4	6	1.5189	0.1873	1.5182
No log	0.5333	8	1.0415	0.1741	1.0415
No log	0.6667	10	1.3450	0.2327	1.3453
No log	0.8	12	2.7213	0.1379	2.7215
No log	0.9333	14	1.9117	0.2036	1.9120
No log	1.0667	16	1.0551	0.2973	1.0556
No log	1.2	18	0.8085	0.4178	0.8090
No log	1.3333	20	0.8599	0.4235	0.8605
No log	1.4667	22	1.2293	0.3473	1.2300
No log	1.6	24	1.4481	0.3228	1.4490
No log	1.7333	26	1.0749	0.4124	1.0757
No log	1.8667	28	0.8047	0.4812	0.8054
No log	2.0	30	0.7416	0.5115	0.7424
No log	2.1333	32	0.8219	0.4458	0.8228
No log	2.2667	34	1.0031	0.4070	1.0041
No log	2.4	36	1.1303	0.4041	1.1312
No log	2.5333	38	1.1323	0.4064	1.1331
No log	2.6667	40	1.0716	0.4240	1.0724
No log	2.8	42	0.8847	0.4894	0.8855
No log	2.9333	44	0.8007	0.5087	0.8014
No log	3.0667	46	0.7579	0.5083	0.7586
No log	3.2	48	0.9311	0.4512	0.9317
No log	3.3333	50	1.0232	0.4313	1.0237
No log	3.4667	52	0.8885	0.4592	0.8890
No log	3.6	54	0.8515	0.4675	0.8520
No log	3.7333	56	0.8352	0.4824	0.8355
No log	3.8667	58	0.8884	0.4855	0.8887
No log	4.0	60	1.0524	0.4439	1.0527
No log	4.1333	62	1.1255	0.4180	1.1260
No log	4.2667	64	0.9389	0.4769	0.9395
No log	4.4	66	0.8175	0.5051	0.8181
No log	4.5333	68	0.8469	0.5060	0.8476
No log	4.6667	70	0.9671	0.4843	0.9678
No log	4.8	72	0.9258	0.4895	0.9264
No log	4.9333	74	0.8740	0.4900	0.8746
No log	5.0667	76	0.8529	0.4874	0.8535
No log	5.2	78	0.9148	0.4757	0.9154
No log	5.3333	80	0.9168	0.4791	0.9175
No log	5.4667	82	0.7903	0.4954	0.7910
No log	5.6	84	0.7057	0.5514	0.7065
No log	5.7333	86	0.7449	0.5282	0.7458
No log	5.8667	88	0.8859	0.4980	0.8867
No log	6.0	90	1.0776	0.4447	1.0784
No log	6.1333	92	1.0638	0.4471	1.0646
No log	6.2667	94	0.8896	0.4741	0.8904
No log	6.4	96	0.7857	0.5028	0.7865
No log	6.5333	98	0.7723	0.5134	0.7731
No log	6.6667	100	0.8110	0.5025	0.8118
No log	6.8	102	0.8781	0.4840	0.8789
No log	6.9333	104	0.9719	0.4805	0.9727
No log	7.0667	106	1.0214	0.4585	1.0221
No log	7.2	108	1.0030	0.4702	1.0037
No log	7.3333	110	0.8786	0.5008	0.8793
No log	7.4667	112	0.8115	0.5083	0.8122
No log	7.6	114	0.8162	0.5036	0.8169
No log	7.7333	116	0.8235	0.5011	0.8242
No log	7.8667	118	0.8121	0.5015	0.8128
No log	8.0	120	0.7687	0.5116	0.7694
No log	8.1333	122	0.7624	0.5101	0.7631
No log	8.2667	124	0.8069	0.5054	0.8075
No log	8.4	126	0.8526	0.5050	0.8532
No log	8.5333	128	0.8600	0.5050	0.8606
No log	8.6667	130	0.8586	0.5055	0.8593
No log	8.8	132	0.8700	0.5019	0.8707
No log	8.9333	134	0.8744	0.5019	0.8750
No log	9.0667	136	0.8880	0.4899	0.8886
No log	9.2	138	0.8928	0.4840	0.8934
No log	9.3333	140	0.8662	0.4960	0.8668
No log	9.4667	142	0.8312	0.5059	0.8318
No log	9.6	144	0.8041	0.5063	0.8048
No log	9.7333	146	0.7847	0.5136	0.7854
No log	9.8667	148	0.7768	0.5125	0.7775
No log	10.0	150	0.7744	0.5129	0.7751

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1