arabert_cross_vocabulary_task5_fold5

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3339
• Qwk: 0.8270
• Mse: 0.3339

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0351	2	3.7424	0.0021	3.7424
No log	0.0702	4	1.7368	0.2488	1.7368
No log	0.1053	6	0.8992	0.2497	0.8992
No log	0.1404	8	0.7327	0.6064	0.7327
No log	0.1754	10	0.8042	0.5559	0.8042
No log	0.2105	12	0.9939	0.6771	0.9939
No log	0.2456	14	0.9616	0.6986	0.9616
No log	0.2807	16	0.8004	0.6308	0.8004
No log	0.3158	18	0.6373	0.6016	0.6373
No log	0.3509	20	0.6361	0.6470	0.6361
No log	0.3860	22	0.5967	0.7168	0.5967
No log	0.4211	24	0.4872	0.7622	0.4872
No log	0.4561	26	0.4462	0.7864	0.4462
No log	0.4912	28	0.4171	0.8029	0.4171
No log	0.5263	30	0.3844	0.8207	0.3844
No log	0.5614	32	0.4099	0.8533	0.4099
No log	0.5965	34	0.4527	0.8592	0.4527
No log	0.6316	36	0.4782	0.8573	0.4782
No log	0.6667	38	0.4629	0.8620	0.4629
No log	0.7018	40	0.3818	0.8464	0.3818
No log	0.7368	42	0.3245	0.8055	0.3245
No log	0.7719	44	0.3187	0.7617	0.3187
No log	0.8070	46	0.3176	0.7494	0.3176
No log	0.8421	48	0.3184	0.7414	0.3184
No log	0.8772	50	0.3131	0.7715	0.3131
No log	0.9123	52	0.3167	0.8053	0.3167
No log	0.9474	54	0.3273	0.8208	0.3273
No log	0.9825	56	0.3339	0.8270	0.3339

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1