edyfjm07/distilbert-base-uncased-v2-finetuned-squad-es

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0170
• Train End Logits Accuracy: 0.9975
• Train Start Logits Accuracy: 0.9950
• Validation Loss: 0.6848
• Validation End Logits Accuracy: 0.8922
• Validation Start Logits Accuracy: 0.8848
• Epoch: 49

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 5000, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Train End Logits Accuracy	Train Start Logits Accuracy	Validation Loss	Validation End Logits Accuracy	Validation Start Logits Accuracy	Epoch
2.4937	0.4238	0.3650	1.6575	0.4535	0.5688	0
1.1993	0.625	0.6425	0.8766	0.6952	0.6840	1
0.7478	0.7262	0.7462	0.7438	0.7323	0.7286	2
0.6099	0.7700	0.7763	0.6805	0.7361	0.7286	3
0.4741	0.8163	0.8263	0.5590	0.8104	0.7658	4
0.4413	0.8263	0.8138	0.6294	0.7955	0.7918	5
0.4165	0.8450	0.8388	0.5712	0.8030	0.7918	6
0.3614	0.8625	0.8525	0.5701	0.8141	0.7695	7
0.3260	0.8737	0.8788	0.6174	0.8216	0.7807	8
0.3187	0.875	0.8687	0.5824	0.8216	0.7955	9
0.2739	0.9050	0.8825	0.5829	0.8216	0.8067	10
0.2465	0.9087	0.9087	0.5796	0.8216	0.8104	11
0.2507	0.8950	0.8913	0.6048	0.8587	0.7881	12
0.2102	0.9225	0.9075	0.5560	0.8662	0.8253	13
0.2129	0.9187	0.9137	0.5616	0.8439	0.8439	14
0.1939	0.9237	0.9225	0.5186	0.8587	0.8439	15
0.1621	0.9400	0.9413	0.5331	0.8587	0.8476	16
0.1620	0.9463	0.9463	0.5752	0.8550	0.8513	17
0.1450	0.9463	0.9362	0.5934	0.8699	0.8476	18
0.1374	0.9400	0.9525	0.5648	0.8699	0.8625	19
0.1234	0.9438	0.9488	0.6096	0.8848	0.8327	20
0.1300	0.9525	0.9613	0.5854	0.8699	0.8625	21
0.1095	0.9600	0.9513	0.5962	0.8662	0.8587	22
0.1168	0.9588	0.9588	0.6229	0.8736	0.8513	23
0.0919	0.9650	0.9638	0.6139	0.8773	0.8699	24
0.0880	0.9725	0.9700	0.6668	0.8699	0.8401	25
0.0828	0.9725	0.9600	0.6261	0.8699	0.8550	26
0.0846	0.9675	0.9725	0.7065	0.8662	0.8662	27
0.0833	0.9725	0.9638	0.6470	0.8699	0.8662	28
0.0772	0.9787	0.9688	0.6112	0.8810	0.8922	29
0.0465	0.9837	0.9837	0.6582	0.8699	0.8736	30
0.0619	0.9700	0.9800	0.6287	0.8810	0.8736	31
0.0589	0.9800	0.9775	0.6796	0.8736	0.8625	32
0.0446	0.9862	0.9825	0.6717	0.8848	0.8699	33
0.0401	0.9862	0.9837	0.6632	0.8848	0.8848	34
0.0432	0.9800	0.9887	0.6478	0.8773	0.8736	35
0.0406	0.9837	0.9862	0.6627	0.8773	0.8810	36
0.0392	0.9837	0.9875	0.6827	0.8848	0.8699	37
0.0351	0.9825	0.9912	0.6693	0.8810	0.8699	38
0.0308	0.9912	0.9900	0.6689	0.8810	0.8810	39
0.0303	0.9850	0.9912	0.7091	0.8922	0.8699	40
0.0334	0.9937	0.9850	0.6542	0.8885	0.8810	41
0.0346	0.9912	0.9850	0.6472	0.8885	0.8736	42
0.0264	0.9912	0.9925	0.6369	0.8885	0.8848	43
0.0261	0.9937	0.9912	0.6484	0.8885	0.8810	44
0.0255	0.9912	0.9937	0.6768	0.8885	0.8773	45
0.0223	0.9912	0.9925	0.6858	0.8922	0.8848	46
0.0254	0.9937	0.9925	0.6755	0.8922	0.8885	47
0.0208	0.9962	0.9900	0.6838	0.8922	0.8848	48
0.0170	0.9975	0.9950	0.6848	0.8922	0.8848	49

Framework versions

• Transformers 4.40.2
• TensorFlow 2.15.0
• Datasets 2.19.1
• Tokenizers 0.19.1