vit5-base-vietnews-summarization-sport

This model is a fine-tuned version of VietAI/vit5-base-vietnews-summarization on an toanduc/vietnamese-sport-newspapers-summarization dataset. It achieves the following results on the evaluation set:

• Loss: 1.7541
• Rouge1: 24.4342
• Rouge2: 12.2779
• Rougel: 18.7382
• Rougelsum: 21.4735
• Gen Len: 19.0

Model description

Summary model for Vietnamese sports articles under the transfer tag

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 16
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.7931	1.0	2897	1.6246	25.1902	11.9842	18.7125	21.8765	19.0
1.5869	2.0	5794	1.5557	24.8104	12.918	19.1867	21.9091	19.0
1.4568	3.0	8691	1.5264	24.3498	12.6452	18.7426	21.4748	19.0
1.3375	4.0	11588	1.5259	24.3584	12.2843	18.7177	21.4296	19.0
1.2417	5.0	14485	1.5299	24.3633	11.9901	18.5573	21.4541	19.0
1.1705	6.0	17382	1.5533	24.4314	12.3337	18.7517	21.5669	19.0
1.0965	7.0	20279	1.5716	24.7562	12.4883	18.8926	21.7281	19.0
1.0479	8.0	23176	1.5918	24.3464	12.4018	18.6725	21.3891	19.0
0.9993	9.0	26073	1.6116	24.6254	12.1213	18.7091	21.5747	19.0
0.9512	10.0	28970	1.6427	24.6133	12.4088	18.8244	21.6162	19.0
0.8903	11.0	31867	1.6595	24.4154	12.4455	18.721	21.4605	19.0
0.8463	12.0	34764	1.6953	24.6333	12.2631	18.8074	21.6077	19.0
0.8224	13.0	37661	1.7127	24.2227	12.141	18.5775	21.321	19.0
0.8003	14.0	40558	1.7348	24.3755	12.1357	18.6279	21.383	19.0
0.7771	15.0	43455	1.7455	24.4422	12.1864	18.7155	21.4499	19.0
0.7666	16.0	46352	1.7541	24.4342	12.2779	18.7382	21.4735	19.0

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2