arabert_cross_vocabulary_task4_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5474
• Qwk: 0.4497
• Mse: 0.5474

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0308	2	8.1440	-0.0005	8.1440
No log	0.0615	4	4.7743	0.0021	4.7743
No log	0.0923	6	2.7118	0.0416	2.7118
No log	0.1231	8	1.7150	0.0799	1.7150
No log	0.1538	10	0.8541	0.1088	0.8541
No log	0.1846	12	0.8601	0.1094	0.8601
No log	0.2154	14	0.8470	0.1589	0.8470
No log	0.2462	16	1.0145	0.1789	1.0145
No log	0.2769	18	1.4913	0.1666	1.4913
No log	0.3077	20	1.4930	0.2073	1.4930
No log	0.3385	22	0.8055	0.3243	0.8055
No log	0.3692	24	0.5508	0.4354	0.5508
No log	0.4	26	0.6031	0.3790	0.6031
No log	0.4308	28	0.5860	0.4121	0.5860
No log	0.4615	30	0.6336	0.4190	0.6336
No log	0.4923	32	0.9905	0.3071	0.9905
No log	0.5231	34	1.0889	0.2932	1.0889
No log	0.5538	36	0.8745	0.3389	0.8745
No log	0.5846	38	0.7332	0.3681	0.7332
No log	0.6154	40	0.6486	0.3927	0.6486
No log	0.6462	42	0.6347	0.3914	0.6347
No log	0.6769	44	0.6439	0.3923	0.6439
No log	0.7077	46	0.6224	0.4042	0.6224
No log	0.7385	48	0.6224	0.4075	0.6224
No log	0.7692	50	0.5740	0.4350	0.5740
No log	0.8	52	0.5606	0.4443	0.5606
No log	0.8308	54	0.5580	0.4473	0.5580
No log	0.8615	56	0.5443	0.4451	0.5443
No log	0.8923	58	0.5430	0.4451	0.5430
No log	0.9231	60	0.5370	0.4575	0.5370
No log	0.9538	62	0.5412	0.4517	0.5412
No log	0.9846	64	0.5474	0.4497	0.5474

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1