arabert_cross_vocabulary_task1_fold6

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4691
• Qwk: 0.6043
• Mse: 0.4695

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1333	2	3.5366	0.0172	3.5407
No log	0.2667	4	1.9631	0.0854	1.9633
No log	0.4	6	0.8549	0.3315	0.8566
No log	0.5333	8	0.8207	0.4747	0.8223
No log	0.6667	10	0.8985	0.3094	0.8995
No log	0.8	12	0.7171	0.3820	0.7187
No log	0.9333	14	0.4768	0.6161	0.4790
No log	1.0667	16	0.4681	0.5385	0.4699
No log	1.2	18	0.4222	0.6122	0.4237
No log	1.3333	20	0.4873	0.6466	0.4880
No log	1.4667	22	0.7664	0.5029	0.7657
No log	1.6	24	0.5655	0.6403	0.5656
No log	1.7333	26	0.4131	0.7256	0.4147
No log	1.8667	28	0.4061	0.6758	0.4080
No log	2.0	30	0.4877	0.5770	0.4895
No log	2.1333	32	0.4719	0.5718	0.4737
No log	2.2667	34	0.3880	0.6223	0.3901
No log	2.4	36	0.3630	0.7266	0.3652
No log	2.5333	38	0.3863	0.7041	0.3880
No log	2.6667	40	0.4796	0.6361	0.4807
No log	2.8	42	0.4544	0.6439	0.4556
No log	2.9333	44	0.3815	0.6948	0.3833
No log	3.0667	46	0.3666	0.7341	0.3685
No log	3.2	48	0.3991	0.6331	0.4003
No log	3.3333	50	0.5108	0.5581	0.5116
No log	3.4667	52	0.4518	0.5955	0.4529
No log	3.6	54	0.3551	0.7316	0.3569
No log	3.7333	56	0.3567	0.7709	0.3588
No log	3.8667	58	0.3551	0.7325	0.3569
No log	4.0	60	0.4528	0.6083	0.4540
No log	4.1333	62	0.5123	0.5776	0.5133
No log	4.2667	64	0.4206	0.6489	0.4217
No log	4.4	66	0.3971	0.6953	0.3982
No log	4.5333	68	0.3984	0.6877	0.3995
No log	4.6667	70	0.4585	0.6205	0.4594
No log	4.8	72	0.4840	0.5808	0.4849
No log	4.9333	74	0.3955	0.6703	0.3964
No log	5.0667	76	0.3740	0.7160	0.3752
No log	5.2	78	0.3739	0.6999	0.3750
No log	5.3333	80	0.4368	0.6324	0.4377
No log	5.4667	82	0.5136	0.5929	0.5141
No log	5.6	84	0.4855	0.6019	0.4859
No log	5.7333	86	0.4397	0.6381	0.4401
No log	5.8667	88	0.4409	0.6371	0.4412
No log	6.0	90	0.4781	0.6117	0.4783
No log	6.1333	92	0.5415	0.5949	0.5416
No log	6.2667	94	0.5043	0.6038	0.5046
No log	6.4	96	0.4420	0.6502	0.4425
No log	6.5333	98	0.4236	0.6652	0.4242
No log	6.6667	100	0.4415	0.6391	0.4420
No log	6.8	102	0.5253	0.5827	0.5257
No log	6.9333	104	0.5412	0.5748	0.5416
No log	7.0667	106	0.4800	0.6034	0.4805
No log	7.2	108	0.4525	0.6318	0.4532
No log	7.3333	110	0.4582	0.6290	0.4589
No log	7.4667	112	0.4760	0.6084	0.4767
No log	7.6	114	0.4942	0.5915	0.4948
No log	7.7333	116	0.5342	0.5816	0.5347
No log	7.8667	118	0.5123	0.5954	0.5127
No log	8.0	120	0.4615	0.6280	0.4619
No log	8.1333	122	0.4445	0.6622	0.4449
No log	8.2667	124	0.4551	0.6529	0.4555
No log	8.4	126	0.4888	0.6100	0.4891
No log	8.5333	128	0.5436	0.5912	0.5439
No log	8.6667	130	0.5938	0.5773	0.5940
No log	8.8	132	0.5869	0.5781	0.5872
No log	8.9333	134	0.5432	0.5847	0.5436
No log	9.0667	136	0.4923	0.6054	0.4927
No log	9.2	138	0.4568	0.6321	0.4573
No log	9.3333	140	0.4453	0.6608	0.4458
No log	9.4667	142	0.4465	0.6608	0.4470
No log	9.6	144	0.4542	0.6387	0.4547
No log	9.7333	146	0.4614	0.6167	0.4619
No log	9.8667	148	0.4677	0.6043	0.4682
No log	10.0	150	0.4691	0.6043	0.4695

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1