mt5-base-finetuned-sumeczech

This model is a fine-tuned version of google/mt5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9291
• Rouge1: 15.9842
• Rouge2: 5.0275
• Rougel: 12.6308
• Rougelsum: 14.0073

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
2.6446	1.0	108450	2.4043	13.4797	3.1596	10.6012	11.798
2.3864	2.0	216900	2.3327	13.955	3.387	10.9208	12.165
2.3381	3.0	325350	2.2699	14.2671	3.5872	11.1539	12.4443
2.2583	4.0	433800	2.2085	14.5162	3.9249	11.4167	12.697
2.178	5.0	542250	2.1429	14.8376	4.1524	11.6426	12.9856
2.0847	6.0	650700	2.0678	15.0717	4.3497	11.8584	13.1779
1.9676	7.0	759150	1.9866	15.7074	4.7106	12.3935	13.7652
1.8196	8.0	867600	1.9291	15.9842	5.0275	12.6308	14.0073

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1