--- base_model: aubmindlab/bert-base-arabertv02 tags: - generated_from_trainer model-index: - name: arabert_cross_relevance_task7_fold3 results: [] --- # arabert_cross_relevance_task7_fold3 This model is a fine-tuned version of [aubmindlab/bert-base-arabertv02](https://huggingface.co/aubmindlab/bert-base-arabertv02) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.4851 - Qwk: 0.0152 - Mse: 0.4851 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 64 - eval_batch_size: 64 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 10 ### Training results | Training Loss | Epoch | Step | Validation Loss | Qwk | Mse | |:-------------:|:------:|:----:|:---------------:|:-------:|:------:| | No log | 0.1176 | 2 | 0.4468 | 0.0241 | 0.4468 | | No log | 0.2353 | 4 | 0.5293 | 0.1111 | 0.5293 | | No log | 0.3529 | 6 | 0.4293 | 0.0657 | 0.4293 | | No log | 0.4706 | 8 | 0.2782 | 0.0 | 0.2782 | | No log | 0.5882 | 10 | 0.2775 | 0.0 | 0.2775 | | No log | 0.7059 | 12 | 0.3091 | 0.0172 | 0.3091 | | No log | 0.8235 | 14 | 0.3313 | -0.0714 | 0.3313 | | No log | 0.9412 | 16 | 0.3259 | 0.0101 | 0.3259 | | No log | 1.0588 | 18 | 0.3382 | 0.0 | 0.3382 | | No log | 1.1765 | 20 | 0.3574 | -0.0135 | 0.3574 | | No log | 1.2941 | 22 | 0.3580 | -0.0274 | 0.3580 | | No log | 1.4118 | 24 | 0.4301 | 0.0 | 0.4301 | | No log | 1.5294 | 26 | 0.4619 | 0.0 | 0.4619 | | No log | 1.6471 | 28 | 0.3492 | 0.0 | 0.3492 | | No log | 1.7647 | 30 | 0.3168 | -0.0417 | 0.3168 | | No log | 1.8824 | 32 | 0.3350 | -0.0496 | 0.3350 | | No log | 2.0 | 34 | 0.2965 | 0.0203 | 0.2965 | | No log | 2.1176 | 36 | 0.2856 | 0.0 | 0.2856 | | No log | 2.2353 | 38 | 0.2924 | 0.0 | 0.2924 | | No log | 2.3529 | 40 | 0.2917 | 0.0 | 0.2917 | | No log | 2.4706 | 42 | 0.2928 | 0.0 | 0.2928 | | No log | 2.5882 | 44 | 0.2979 | -0.0616 | 0.2979 | | No log | 2.7059 | 46 | 0.3126 | 0.0 | 0.3126 | | No log | 2.8235 | 48 | 0.3738 | 0.0 | 0.3738 | | No log | 2.9412 | 50 | 0.3664 | 0.0 | 0.3664 | | No log | 3.0588 | 52 | 0.3323 | 0.0 | 0.3323 | | No log | 3.1765 | 54 | 0.3388 | 0.0 | 0.3388 | | No log | 3.2941 | 56 | 0.3765 | 0.0 | 0.3765 | | No log | 3.4118 | 58 | 0.3896 | 0.0 | 0.3896 | | No log | 3.5294 | 60 | 0.3597 | 0.0 | 0.3597 | | No log | 3.6471 | 62 | 0.3248 | 0.0 | 0.3248 | | No log | 3.7647 | 64 | 0.3130 | -0.0473 | 0.3130 | | No log | 3.8824 | 66 | 0.3039 | 0.0 | 0.3039 | | No log | 4.0 | 68 | 0.3036 | -0.0235 | 0.3036 | | No log | 4.1176 | 70 | 0.3005 | 0.0 | 0.3005 | | No log | 4.2353 | 72 | 0.3729 | 0.0 | 0.3729 | | No log | 4.3529 | 74 | 0.4274 | 0.0 | 0.4274 | | No log | 4.4706 | 76 | 0.4149 | 0.0 | 0.4149 | | No log | 4.5882 | 78 | 0.3636 | 0.0 | 0.3636 | | No log | 4.7059 | 80 | 0.3435 | 0.0 | 0.3435 | | No log | 4.8235 | 82 | 0.3605 | 0.0 | 0.3605 | | No log | 4.9412 | 84 | 0.4118 | 0.0 | 0.4118 | | No log | 5.0588 | 86 | 0.4373 | 0.0 | 0.4373 | | No log | 5.1765 | 88 | 0.3877 | 0.0 | 0.3877 | | No log | 5.2941 | 90 | 0.3152 | 0.0 | 0.3152 | | No log | 5.4118 | 92 | 0.3033 | 0.0 | 0.3033 | | No log | 5.5294 | 94 | 0.3085 | 0.0 | 0.3085 | | No log | 5.6471 | 96 | 0.3451 | 0.0 | 0.3451 | | No log | 5.7647 | 98 | 0.4554 | 0.0 | 0.4554 | | No log | 5.8824 | 100 | 0.5290 | 0.0230 | 0.5290 | | No log | 6.0 | 102 | 0.5208 | 0.0230 | 0.5208 | | No log | 6.1176 | 104 | 0.4384 | 0.0 | 0.4384 | | No log | 6.2353 | 106 | 0.3592 | 0.0 | 0.3592 | | No log | 6.3529 | 108 | 0.3270 | -0.0235 | 0.3270 | | No log | 6.4706 | 110 | 0.3341 | -0.0235 | 0.3341 | | No log | 6.5882 | 112 | 0.3501 | 0.0 | 0.3501 | | No log | 6.7059 | 114 | 0.4010 | 0.0224 | 0.4010 | | No log | 6.8235 | 116 | 0.4289 | 0.0224 | 0.4289 | | No log | 6.9412 | 118 | 0.4617 | 0.0432 | 0.4617 | | No log | 7.0588 | 120 | 0.4638 | 0.0432 | 0.4638 | | No log | 7.1765 | 122 | 0.4444 | 0.0432 | 0.4444 | | No log | 7.2941 | 124 | 0.4625 | 0.0120 | 0.4625 | | No log | 7.4118 | 126 | 0.5041 | 0.0139 | 0.5041 | | No log | 7.5294 | 128 | 0.5111 | 0.0139 | 0.5111 | | No log | 7.6471 | 130 | 0.5193 | -0.0026 | 0.5193 | | No log | 7.7647 | 132 | 0.4942 | 0.0139 | 0.4942 | | No log | 7.8824 | 134 | 0.4487 | 0.0432 | 0.4487 | | No log | 8.0 | 136 | 0.3941 | 0.0 | 0.3941 | | No log | 8.1176 | 138 | 0.3656 | 0.0 | 0.3656 | | No log | 8.2353 | 140 | 0.3670 | 0.0 | 0.3670 | | No log | 8.3529 | 142 | 0.3813 | 0.0 | 0.3813 | | No log | 8.4706 | 144 | 0.4173 | 0.0432 | 0.4173 | | No log | 8.5882 | 146 | 0.4510 | 0.0230 | 0.4510 | | No log | 8.7059 | 148 | 0.4637 | 0.0230 | 0.4637 | | No log | 8.8235 | 150 | 0.4865 | -0.0026 | 0.4865 | | No log | 8.9412 | 152 | 0.5054 | -0.0101 | 0.5054 | | No log | 9.0588 | 154 | 0.5000 | 0.0074 | 0.5000 | | No log | 9.1765 | 156 | 0.4847 | -0.0026 | 0.4847 | | No log | 9.2941 | 158 | 0.4728 | -0.0026 | 0.4728 | | No log | 9.4118 | 160 | 0.4652 | 0.0054 | 0.4652 | | No log | 9.5294 | 162 | 0.4641 | 0.0054 | 0.4641 | | No log | 9.6471 | 164 | 0.4713 | -0.0026 | 0.4713 | | No log | 9.7647 | 166 | 0.4801 | 0.0152 | 0.4801 | | No log | 9.8824 | 168 | 0.4839 | 0.0152 | 0.4839 | | No log | 10.0 | 170 | 0.4851 | 0.0152 | 0.4851 | ### Framework versions - Transformers 4.44.0 - Pytorch 2.4.0 - Datasets 2.21.0 - Tokenizers 0.19.1