edyfjm07/distilbert-base-uncased-TIC2-finetuned-squad-es

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0266
• Train End Logits Accuracy: 0.9905
• Train Start Logits Accuracy: 0.9905
• Validation Loss: 0.9270
• Validation End Logits Accuracy: 0.8777
• Validation Start Logits Accuracy: 0.8276
• Epoch: 50

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 6069, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Train End Logits Accuracy	Train Start Logits Accuracy	Validation Loss	Validation End Logits Accuracy	Validation Start Logits Accuracy	Epoch
2.9957	0.3246	0.2973	1.6002	0.5078	0.5987	0
1.3461	0.5819	0.6387	0.9536	0.6991	0.6928	1
0.8784	0.7059	0.7374	0.8048	0.7492	0.7273	2
0.7023	0.7521	0.7679	0.7546	0.7555	0.7367	3
0.5582	0.7962	0.8067	0.7055	0.7680	0.7962	4
0.4846	0.8057	0.8456	0.6709	0.7774	0.7994	5
0.3972	0.8246	0.8739	0.7190	0.8088	0.8150	6
0.3642	0.8456	0.8813	0.6769	0.8401	0.8150	7
0.2893	0.8561	0.9044	0.6974	0.8119	0.7994	8
0.2928	0.8708	0.9097	0.6651	0.8464	0.8245	9
0.2499	0.8960	0.9160	0.7726	0.8307	0.8056	10
0.2193	0.9149	0.9286	0.7103	0.8339	0.8182	11
0.1987	0.9212	0.9328	0.6805	0.8182	0.8245	12
0.1918	0.9212	0.9359	0.7566	0.8339	0.7931	13
0.1657	0.9286	0.9454	0.7386	0.8433	0.8056	14
0.1440	0.9422	0.9485	0.6785	0.8589	0.8119	15
0.1543	0.9328	0.9527	0.7059	0.8652	0.7962	16
0.1270	0.9527	0.9538	0.8083	0.8527	0.8025	17
0.1107	0.9580	0.9548	0.7088	0.8683	0.8307	18
0.1173	0.9527	0.9622	0.7848	0.8527	0.7931	19
0.0964	0.9643	0.9664	0.8175	0.8621	0.8119	20
0.0986	0.9674	0.9643	0.8027	0.8621	0.8088	21
0.0976	0.9590	0.9601	0.8114	0.8621	0.8213	22
0.0784	0.9664	0.9748	0.8268	0.8652	0.8182	23
0.0707	0.9706	0.9716	0.8681	0.8527	0.8025	24
0.0735	0.9685	0.9727	0.8315	0.8652	0.8119	25
0.0679	0.9727	0.9716	0.8746	0.8495	0.8056	26
0.0556	0.9727	0.9737	0.8374	0.8527	0.8119	27
0.0601	0.9716	0.9800	0.8785	0.8527	0.7900	28
0.0787	0.9674	0.9632	0.8050	0.8683	0.8056	29
0.0570	0.9716	0.9779	0.8419	0.8652	0.8088	30
0.0567	0.9769	0.9790	0.8142	0.8715	0.8213	31
0.0520	0.9790	0.9853	0.8440	0.8715	0.8182	32
0.0444	0.9811	0.9832	0.8456	0.8777	0.8182	33
0.0500	0.9748	0.9832	0.8039	0.8777	0.8088	34
0.0480	0.9800	0.9811	0.8078	0.8558	0.8182	35
0.0434	0.9821	0.9821	0.8186	0.8621	0.8276	36
0.0406	0.9842	0.9842	0.8314	0.8715	0.8245	37
0.0395	0.9842	0.9842	0.8117	0.8840	0.8307	38
0.0374	0.9832	0.9874	0.8603	0.8840	0.8276	39
0.0279	0.9884	0.9905	0.8761	0.8934	0.8307	40
0.0329	0.9842	0.9895	0.8870	0.8903	0.8307	41
0.0289	0.9863	0.9874	0.8906	0.8903	0.8307	42
0.0308	0.9853	0.9863	0.9079	0.8840	0.8182	43
0.0311	0.9853	0.9874	0.9111	0.8871	0.8307	44
0.0334	0.9842	0.9832	0.9121	0.8903	0.8276	45
0.0266	0.9874	0.9937	0.9113	0.8871	0.8307	46
0.0310	0.9842	0.9905	0.9117	0.8840	0.8276	47
0.0263	0.9884	0.9926	0.9192	0.8871	0.8307	48
0.0254	0.9874	0.9905	0.9257	0.8777	0.8245	49
0.0266	0.9905	0.9905	0.9270	0.8777	0.8276	50

Framework versions

• Transformers 4.41.2
• TensorFlow 2.15.0
• Datasets 2.20.0
• Tokenizers 0.19.1