long-t5-tglobal-base-lora-finetuned

This model is a fine-tuned version of google/long-t5-tglobal-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1044
• Rouge1: 18.5008
• Rouge2: 10.9163
• Rougel: 15.5588
• Rougelsum: 17.0777
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 16

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.7746	1.0	4725	1.1997	16.5663	8.1613	13.4306	15.1181	19.0
1.6751	2.0	9450	1.1665	17.3821	9.5041	14.7744	16.1736	19.0
1.6576	3.0	14175	1.1467	17.7909	10.0228	15.1104	16.3255	19.0
1.6061	4.0	18900	1.1407	17.9449	9.6832	14.8839	16.2538	19.0
1.6107	5.0	23625	1.1311	17.8787	9.6712	14.7942	16.1142	19.0
1.5935	6.0	28350	1.1274	18.1678	10.0074	14.937	16.4508	19.0
1.5649	7.0	33075	1.1178	17.9792	9.915	14.8056	16.2568	19.0
1.5707	8.0	37800	1.1141	18.0138	9.9492	15.0511	16.423	19.0
1.555	9.0	42525	1.1126	17.7218	9.8772	14.8897	16.3123	19.0
1.57	10.0	47250	1.1111	18.0723	10.4508	15.2597	16.6341	19.0
1.5189	11.0	51975	1.1119	18.2068	10.5754	15.3172	16.8677	19.0
1.5619	12.0	56700	1.1074	18.3754	10.6836	15.4124	16.9231	19.0
1.5498	13.0	61425	1.1056	18.3001	10.7064	15.3411	16.9528	19.0
1.552	14.0	66150	1.1028	18.4318	10.8917	15.4457	17.0401	19.0
1.5608	15.0	70875	1.1032	18.4332	10.8393	15.4484	17.0042	19.0
1.5288	16.0	75600	1.1044	18.5008	10.9163	15.5588	17.0777	19.0

Framework versions

• PEFT 0.11.1
• Transformers 4.42.4
• Pytorch 2.2.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1