arabert_cross_vocabulary_task7_fold5

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4813
• Qwk: 0.8159
• Mse: 0.4813

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0328	2	1.7829	0.0226	1.7829
No log	0.0656	4	1.0550	0.3246	1.0550
No log	0.0984	6	1.2058	0.3390	1.2058
No log	0.1311	8	1.2897	0.4716	1.2897
No log	0.1639	10	0.7738	0.6671	0.7738
No log	0.1967	12	0.4801	0.6113	0.4801
No log	0.2295	14	0.4704	0.6859	0.4704
No log	0.2623	16	0.6162	0.7884	0.6162
No log	0.2951	18	0.8233	0.7575	0.8233
No log	0.3279	20	0.8253	0.7631	0.8253
No log	0.3607	22	0.6816	0.7772	0.6816
No log	0.3934	24	0.6169	0.7875	0.6169
No log	0.4262	26	0.6070	0.7969	0.6070
No log	0.4590	28	0.5939	0.7990	0.5939
No log	0.4918	30	0.5668	0.7902	0.5668
No log	0.5246	32	0.5026	0.8069	0.5026
No log	0.5574	34	0.5333	0.7935	0.5333
No log	0.5902	36	0.6348	0.8099	0.6348
No log	0.6230	38	0.7737	0.7826	0.7737
No log	0.6557	40	0.7722	0.7861	0.7722
No log	0.6885	42	0.6762	0.7874	0.6762
No log	0.7213	44	0.5498	0.8024	0.5498
No log	0.7541	46	0.4936	0.8014	0.4936
No log	0.7869	48	0.4728	0.8025	0.4728
No log	0.8197	50	0.4592	0.8048	0.4592
No log	0.8525	52	0.4339	0.8081	0.4339
No log	0.8852	54	0.4330	0.8048	0.4330
No log	0.9180	56	0.4495	0.8048	0.4495
No log	0.9508	58	0.4689	0.8114	0.4689
No log	0.9836	60	0.4813	0.8159	0.4813

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1