arabert_cross_development_task1_fold4

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3891
• Qwk: 0.7487
• Mse: 0.3891

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.125	2	2.7046	0.0134	2.7046
No log	0.25	4	1.3006	0.1103	1.3006
No log	0.375	6	0.7355	0.3354	0.7355
No log	0.5	8	0.8932	0.3820	0.8932
No log	0.625	10	0.5839	0.4676	0.5839
No log	0.75	12	0.4634	0.5294	0.4634
No log	0.875	14	0.4578	0.5335	0.4578
No log	1.0	16	0.5111	0.5160	0.5111
No log	1.125	18	0.4822	0.6507	0.4822
No log	1.25	20	0.4309	0.6257	0.4309
No log	1.375	22	0.4637	0.6422	0.4637
No log	1.5	24	0.5888	0.7264	0.5888
No log	1.625	26	0.5751	0.7173	0.5751
No log	1.75	28	0.4309	0.6445	0.4309
No log	1.875	30	0.3977	0.6113	0.3977
No log	2.0	32	0.4095	0.6330	0.4095
No log	2.125	34	0.4861	0.7147	0.4861
No log	2.25	36	0.5155	0.7516	0.5155
No log	2.375	38	0.4703	0.7321	0.4703
No log	2.5	40	0.3861	0.6978	0.3861
No log	2.625	42	0.3964	0.7189	0.3964
No log	2.75	44	0.5105	0.7660	0.5105
No log	2.875	46	0.5630	0.7439	0.5630
No log	3.0	48	0.4666	0.7758	0.4666
No log	3.125	50	0.4033	0.7314	0.4033
No log	3.25	52	0.3886	0.7225	0.3886
No log	3.375	54	0.4264	0.7369	0.4264
No log	3.5	56	0.4681	0.7538	0.4681
No log	3.625	58	0.4255	0.7357	0.4255
No log	3.75	60	0.3784	0.7381	0.3784
No log	3.875	62	0.3835	0.7261	0.3835
No log	4.0	64	0.3863	0.7091	0.3863
No log	4.125	66	0.3964	0.7022	0.3964
No log	4.25	68	0.4674	0.7519	0.4674
No log	4.375	70	0.5670	0.7310	0.5670
No log	4.5	72	0.5082	0.7265	0.5082
No log	4.625	74	0.3989	0.7387	0.3989
No log	4.75	76	0.3568	0.7218	0.3568
No log	4.875	78	0.3670	0.7343	0.3670
No log	5.0	80	0.4147	0.7453	0.4147
No log	5.125	82	0.4613	0.7583	0.4613
No log	5.25	84	0.4365	0.7493	0.4365
No log	5.375	86	0.3787	0.7383	0.3787
No log	5.5	88	0.3637	0.7327	0.3637
No log	5.625	90	0.3896	0.7461	0.3896
No log	5.75	92	0.4827	0.7585	0.4827
No log	5.875	94	0.5207	0.7560	0.5207
No log	6.0	96	0.4771	0.7622	0.4771
No log	6.125	98	0.4131	0.7595	0.4131
No log	6.25	100	0.3861	0.7447	0.3861
No log	6.375	102	0.3770	0.7473	0.3770
No log	6.5	104	0.4030	0.7421	0.4030
No log	6.625	106	0.4334	0.7447	0.4334
No log	6.75	108	0.4677	0.7616	0.4677
No log	6.875	110	0.4931	0.7670	0.4931
No log	7.0	112	0.4703	0.7622	0.4703
No log	7.125	114	0.4736	0.7622	0.4736
No log	7.25	116	0.4565	0.7580	0.4565
No log	7.375	118	0.4114	0.7521	0.4114
No log	7.5	120	0.3925	0.7534	0.3925
No log	7.625	122	0.3937	0.7441	0.3937
No log	7.75	124	0.3906	0.7441	0.3906
No log	7.875	126	0.3958	0.7488	0.3958
No log	8.0	128	0.4005	0.7481	0.4005
No log	8.125	130	0.4095	0.7399	0.4095
No log	8.25	132	0.4044	0.7426	0.4044
No log	8.375	134	0.3885	0.7487	0.3885
No log	8.5	136	0.3727	0.7455	0.3727
No log	8.625	138	0.3751	0.7428	0.3751
No log	8.75	140	0.3851	0.7477	0.3851
No log	8.875	142	0.4024	0.7472	0.4024
No log	9.0	144	0.4118	0.7505	0.4118
No log	9.125	146	0.4171	0.7505	0.4171
No log	9.25	148	0.4179	0.7495	0.4179
No log	9.375	150	0.4096	0.7483	0.4096
No log	9.5	152	0.4015	0.7555	0.4015
No log	9.625	154	0.3949	0.7508	0.3949
No log	9.75	156	0.3905	0.7487	0.3905
No log	9.875	158	0.3894	0.7487	0.3894
No log	10.0	160	0.3891	0.7487	0.3891

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1