Monglish_Arabic_FAQ

This model is a fine-tuned version of CAMeL-Lab/bert-base-arabic-camelbert-msa on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0223

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss
2.1357	1.0	20	2.0505
1.053	2.0	40	1.4488
0.4438	3.0	60	1.0301
0.5544	4.0	80	0.6261
0.1276	5.0	100	0.4010
0.1003	6.0	120	0.2514
0.1812	7.0	140	0.1628
0.0273	8.0	160	0.1075
0.015	9.0	180	0.0596
0.0374	10.0	200	0.0435
0.03	11.0	220	0.0418
0.0165	12.0	240	0.0398
0.0158	13.0	260	0.0310
0.0142	14.0	280	0.0297
0.006	15.0	300	0.0267
0.0128	16.0	320	0.0259
0.0103	17.0	340	0.0256
0.0163	18.0	360	0.0231
0.0126	19.0	380	0.0225
0.0053	20.0	400	0.0225
0.011	21.0	420	0.0225
0.0161	22.0	440	0.0229
0.0091	23.0	460	0.0230
0.0159	24.0	480	0.0225
0.0131	25.0	500	0.0223

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0+cu121
• Datasets 3.0.0
• Tokenizers 0.19.1