edyfjm07/distilbert-base-uncased-TIC1-finetuned-squad-es

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0173
• Train End Logits Accuracy: 0.9926
• Train Start Logits Accuracy: 0.9937
• Validation Loss: 0.6833
• Validation End Logits Accuracy: 0.8809
• Validation Start Logits Accuracy: 0.8746
• Epoch: 50

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 6069, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Train End Logits Accuracy	Train Start Logits Accuracy	Validation Loss	Validation End Logits Accuracy	Validation Start Logits Accuracy	Epoch
2.9391	0.3582	0.3183	1.5236	0.5611	0.5862	0
1.1532	0.6292	0.6586	0.8407	0.7085	0.7179	1
0.7185	0.7521	0.7563	0.7432	0.7962	0.7555	2
0.6133	0.7763	0.7784	0.6925	0.7712	0.7524	3
0.4777	0.8288	0.8267	0.6963	0.7524	0.7962	4
0.4441	0.8298	0.8403	0.6422	0.8182	0.7806	5
0.3896	0.8519	0.8645	0.6378	0.7900	0.8056	6
0.3642	0.8508	0.8771	0.6286	0.8088	0.7994	7
0.3068	0.8960	0.8887	0.5387	0.8433	0.8558	8
0.2755	0.8845	0.8845	0.6049	0.8245	0.8307	9
0.2711	0.9023	0.9023	0.5653	0.8527	0.8370	10
0.2260	0.9065	0.9202	0.6267	0.8589	0.8150	11
0.2016	0.9086	0.9286	0.6035	0.8777	0.8401	12
0.2044	0.9107	0.9296	0.5305	0.8840	0.8683	13
0.1923	0.9275	0.9296	0.5440	0.8871	0.8621	14
0.1448	0.9307	0.9496	0.5563	0.8934	0.8464	15
0.1465	0.9359	0.9454	0.5626	0.8809	0.8589	16
0.1323	0.9464	0.9517	0.6286	0.8401	0.8495	17
0.1291	0.9506	0.9506	0.5277	0.8746	0.8621	18
0.1156	0.9590	0.9559	0.5341	0.8777	0.8558	19
0.0839	0.9643	0.9748	0.5753	0.8903	0.8527	20
0.1007	0.9538	0.9653	0.5299	0.8746	0.8558	21
0.0901	0.9664	0.9664	0.6034	0.8558	0.8464	22
0.0791	0.9716	0.9779	0.6137	0.8777	0.8495	23
0.0782	0.9653	0.9748	0.6260	0.8809	0.8589	24
0.0747	0.9748	0.9748	0.5973	0.8903	0.8527	25
0.0685	0.9653	0.9821	0.6007	0.8809	0.8777	26
0.0688	0.9685	0.9737	0.5546	0.8903	0.8495	27
0.0513	0.9811	0.9842	0.5925	0.8997	0.8495	28
0.0518	0.9769	0.9863	0.6222	0.8777	0.8746	29
0.0451	0.9748	0.9916	0.6302	0.8777	0.8746	30
0.0424	0.9842	0.9811	0.6389	0.8871	0.8652	31
0.0392	0.9800	0.9853	0.6361	0.8809	0.8715	32
0.0382	0.9842	0.9895	0.6253	0.8840	0.8715	33
0.0405	0.9800	0.9905	0.6734	0.8715	0.8777	34
0.0405	0.9769	0.9905	0.6104	0.8903	0.8652	35
0.0364	0.9790	0.9926	0.6584	0.8809	0.8715	36
0.0272	0.9842	0.9947	0.6439	0.8871	0.8715	37
0.0240	0.9916	0.9937	0.6390	0.8934	0.8746	38
0.0211	0.9884	0.9958	0.6597	0.8871	0.8683	39
0.0277	0.9916	0.9926	0.6561	0.8809	0.8683	40
0.0307	0.9884	0.9874	0.6669	0.8809	0.8652	41
0.0186	0.9947	0.9947	0.6526	0.8871	0.8652	42
0.0178	0.9905	0.9958	0.6681	0.8840	0.8621	43
0.0195	0.9905	0.9926	0.6780	0.8903	0.8683	44
0.0197	0.9937	0.9916	0.7142	0.8777	0.8558	45
0.0176	0.9947	0.9926	0.6914	0.8809	0.8715	46
0.0188	0.9947	0.9916	0.6901	0.8809	0.8652	47
0.0150	0.9958	0.9926	0.6845	0.8809	0.8715	48
0.0170	0.9937	0.9905	0.6826	0.8840	0.8715	49
0.0173	0.9926	0.9937	0.6833	0.8809	0.8746	50

Framework versions

• Transformers 4.41.2
• TensorFlow 2.15.0
• Datasets 2.20.0
• Tokenizers 0.19.1