arabert_cross_vocabulary_task3_fold6

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3859
• Qwk: 0.7465
• Mse: 0.3850

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0299	2	4.8140	0.0	4.7922
No log	0.0597	4	2.6414	0.1002	2.6244
No log	0.0896	6	1.5022	0.0832	1.4837
No log	0.1194	8	0.9313	0.3800	0.9142
No log	0.1493	10	0.8708	0.3343	0.8484
No log	0.1791	12	0.8004	0.4374	0.7791
No log	0.2090	14	0.5474	0.6525	0.5373
No log	0.2388	16	0.4930	0.6276	0.4867
No log	0.2687	18	0.9463	0.4397	0.9380
No log	0.2985	20	0.6703	0.5012	0.6673
No log	0.3284	22	0.4148	0.6772	0.4141
No log	0.3582	24	0.4877	0.7502	0.4873
No log	0.3881	26	0.4802	0.7876	0.4797
No log	0.4179	28	0.4307	0.7387	0.4292
No log	0.4478	30	0.5761	0.5798	0.5713
No log	0.4776	32	0.5786	0.5835	0.5734
No log	0.5075	34	0.4565	0.6502	0.4532
No log	0.5373	36	0.4192	0.7783	0.4180
No log	0.5672	38	0.4672	0.7983	0.4666
No log	0.5970	40	0.4538	0.7680	0.4535
No log	0.6269	42	0.4667	0.7291	0.4667
No log	0.6567	44	0.5942	0.6559	0.5941
No log	0.6866	46	0.6824	0.6019	0.6822
No log	0.7164	48	0.6148	0.6232	0.6147
No log	0.7463	50	0.4953	0.6804	0.4955
No log	0.7761	52	0.4313	0.7294	0.4314
No log	0.8060	54	0.4207	0.7718	0.4207
No log	0.8358	56	0.4290	0.7922	0.4287
No log	0.8657	58	0.4177	0.7922	0.4174
No log	0.8955	60	0.4004	0.7868	0.3999
No log	0.9254	62	0.3904	0.7743	0.3898
No log	0.9552	64	0.3861	0.7613	0.3853
No log	0.9851	66	0.3859	0.7465	0.3850

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1