text_shortening_model_v8

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.3248
• Rouge1: 0.43
• Rouge2: 0.2172
• Rougel: 0.3684
• Rougelsum: 0.3674
• Bert precision: 0.8551
• Bert recall: 0.8369
• Average word count: 9.8214
• Max word count: 17
• Min word count: 5
• Average token count: 15.5857
• % shortened texts with length > 12: 17.1429

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
0.2688	1.0	8	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.284	2.0	16	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.264	3.0	24	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2564	4.0	32	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2727	5.0	40	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2924	6.0	48	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2666	7.0	56	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2662	8.0	64	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2631	9.0	72	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2844	10.0	80	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2653	11.0	88	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2649	12.0	96	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2972	13.0	104	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2553	14.0	112	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.261	15.0	120	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2832	16.0	128	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2635	17.0	136	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2484	18.0	144	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2612	19.0	152	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2996	20.0	160	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2562	21.0	168	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2503	22.0	176	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2763	23.0	184	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2692	24.0	192	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.284	25.0	200	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2838	26.0	208	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2729	27.0	216	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2685	28.0	224	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2599	29.0	232	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2829	30.0	240	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2275	31.0	248	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2605	32.0	256	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2855	33.0	264	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.251	34.0	272	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2629	35.0	280	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2679	36.0	288	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2453	37.0	296	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2853	38.0	304	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2542	39.0	312	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2627	40.0	320	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2668	41.0	328	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2742	42.0	336	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2746	43.0	344	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2768	44.0	352	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2729	45.0	360	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2729	46.0	368	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2788	47.0	376	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.286	48.0	384	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2484	49.0	392	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429
0.2679	50.0	400	2.3248	0.43	0.2172	0.3684	0.3674	0.8551	0.8369	9.8214	17	5	15.5857	17.1429

Framework versions

• Transformers 4.32.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3