mt5-small-finetuned-amazon-en-es

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.9719
• Rouge1: 12.8375
• Rouge2: 4.873
• Rougel: 12.7298
• Rougelsum: 12.7317

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
5.97	1.0	2202	3.2260	9.8577	3.6445	9.7485	9.7848
3.647	2.0	4404	3.0734	11.4598	4.3082	11.3677	11.4138
3.3891	3.0	6606	3.0070	12.0211	4.3801	11.7115	11.7438
3.2655	4.0	8808	2.9878	11.8522	4.3727	11.6773	11.7271
3.1735	5.0	11010	2.9825	12.6673	4.3875	12.5586	12.6202
3.1156	6.0	13212	2.9798	13.0236	5.2414	12.8478	12.9195
3.0672	7.0	15414	2.9771	12.6105	4.8806	12.5344	12.545
3.044	8.0	17616	2.9719	12.8375	4.873	12.7298	12.7317

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3