arabert_cross_vocabulary_task4_fold0

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8406
• Qwk: 0.4869
• Mse: 0.8406

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0308	2	5.8006	0.0028	5.7935
No log	0.0615	4	3.2764	0.0156	3.2707
No log	0.0923	6	1.6565	0.1556	1.6520
No log	0.1231	8	0.9711	0.1893	0.9685
No log	0.1538	10	0.8861	0.2860	0.8838
No log	0.1846	12	1.4789	0.1911	1.4759
No log	0.2154	14	2.1319	0.1487	2.1287
No log	0.2462	16	1.7196	0.2185	1.7170
No log	0.2769	18	1.0076	0.3413	1.0063
No log	0.3077	20	0.8904	0.3996	0.8897
No log	0.3385	22	1.1149	0.3433	1.1142
No log	0.3692	24	1.1193	0.3481	1.1188
No log	0.4	26	0.9820	0.3962	0.9818
No log	0.4308	28	0.8143	0.4684	0.8141
No log	0.4615	30	0.8216	0.4725	0.8215
No log	0.4923	32	1.0728	0.3944	1.0729
No log	0.5231	34	1.2902	0.3621	1.2903
No log	0.5538	36	1.1504	0.3826	1.1506
No log	0.5846	38	0.9402	0.4417	0.9403
No log	0.6154	40	0.8442	0.4861	0.8442
No log	0.6462	42	0.8157	0.5097	0.8157
No log	0.6769	44	0.7981	0.5175	0.7980
No log	0.7077	46	0.8171	0.5105	0.8170
No log	0.7385	48	0.9036	0.4947	0.9036
No log	0.7692	50	0.9337	0.4808	0.9338
No log	0.8	52	0.9430	0.4687	0.9432
No log	0.8308	54	0.9279	0.4743	0.9280
No log	0.8615	56	0.8904	0.4850	0.8905
No log	0.8923	58	0.8610	0.4939	0.8610
No log	0.9231	60	0.8424	0.4894	0.8424
No log	0.9538	62	0.8388	0.4905	0.8388
No log	0.9846	64	0.8406	0.4869	0.8406

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1