flan-t5-sentence-generator

This model is a fine-tuned version of google/flan-t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3271
• Rouge1: 92.6712
• Rouge2: 82.7566
• Rougel: 92.6246
• Rougelsum: 92.5733
• Gen Len: 12.6809

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	38	0.4665	87.5888	72.8489	87.0237	87.1042	11.5745
No log	2.0	76	0.3577	90.7662	79.8453	90.443	90.4784	12.2340
No log	3.0	114	0.3342	92.0014	81.8411	91.999	91.9489	12.4468
No log	4.0	152	0.3343	92.3868	81.5074	92.2937	92.2943	12.5319
No log	5.0	190	0.3517	92.7314	83.1921	92.7259	92.6681	12.7660
No log	6.0	228	0.3271	92.6712	82.7566	92.6246	92.5733	12.6809
No log	7.0	266	0.3285	92.7106	82.4425	92.7382	92.6212	12.6809
No log	8.0	304	0.3379	92.9469	83.0373	92.9539	92.8683	12.6596
No log	9.0	342	0.3318	93.217	83.9024	93.1868	93.1101	12.7234
No log	10.0	380	0.3336	93.0582	83.3947	93.053	92.9652	12.7021

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.15.0