pegasus-large-finetuned-scientific-articles

This model is a fine-tuned version of google/pegasus-large on the scientific_papers dataset. It achieves the following results on the evaluation set:

• Loss: 2.4553
• Rouge1: 32.8743
• Rouge2: 10.8417
• Rougel: 20.3101
• Rougelsum: 28.3673

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.0377	1.0	252	2.5409	30.5637	9.5168	18.2596	26.2196
2.6145	2.0	504	2.4722	31.5518	9.9698	19.9187	26.695
2.4322	3.0	756	2.4553	32.8743	10.8417	20.3101	28.3673

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1