text_shortening_model_v7

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4907
• Rouge1: 0.5855
• Rouge2: 0.3458
• Rougel: 0.525
• Rougelsum: 0.5248
• Bert precision: 0.8932
• Bert recall: 0.9014
• Average word count: 11.6
• Max word count: 18
• Min word count: 6
• Average token count: 16.8

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count
2.285	1.0	8	1.8416	0.5247	0.3056	0.4646	0.4648	0.8769	0.8826	11.2786	18	1	16.7643
1.9309	2.0	16	1.7082	0.5311	0.3091	0.4698	0.4696	0.8779	0.8859	11.6214	18	4	17.1
1.8124	3.0	24	1.6491	0.5348	0.3068	0.4768	0.4763	0.8845	0.8895	11.2071	18	5	16.3357
1.714	4.0	32	1.6132	0.5496	0.3135	0.4871	0.4856	0.8859	0.8931	11.3143	18	5	16.5429
1.6574	5.0	40	1.5831	0.5655	0.3305	0.5051	0.5044	0.887	0.8993	11.8571	17	5	17.2
1.5906	6.0	48	1.5574	0.5706	0.3303	0.5075	0.5071	0.8856	0.902	12.2714	17	6	17.7143
1.5538	7.0	56	1.5241	0.5745	0.3332	0.5096	0.5094	0.8871	0.9011	12.0429	17	5	17.4
1.4875	8.0	64	1.5150	0.5773	0.3353	0.5117	0.512	0.8862	0.9024	12.2	17	6	17.6
1.4466	9.0	72	1.4969	0.5781	0.3345	0.5092	0.5096	0.8881	0.9006	12.0643	17	6	17.3429
1.4166	10.0	80	1.4864	0.5752	0.3326	0.5085	0.5085	0.8887	0.8999	11.9357	17	6	17.2286
1.3887	11.0	88	1.4809	0.5738	0.3271	0.5049	0.5051	0.8862	0.9001	12.1429	17	6	17.4786
1.3321	12.0	96	1.4755	0.5811	0.337	0.5144	0.5145	0.8879	0.9017	12.2429	17	6	17.6286
1.3167	13.0	104	1.4635	0.5816	0.3355	0.5143	0.5137	0.8886	0.9015	12.15	17	6	17.5214
1.2763	14.0	112	1.4593	0.5817	0.3345	0.5141	0.5138	0.8882	0.9007	12.1071	17	6	17.3714
1.2584	15.0	120	1.4640	0.5851	0.337	0.5182	0.5181	0.8884	0.9016	12.15	17	6	17.4143
1.2266	16.0	128	1.4652	0.5777	0.3321	0.5124	0.5127	0.8873	0.9	12.0571	17	6	17.3071
1.2077	17.0	136	1.4627	0.5798	0.3326	0.5142	0.5147	0.8876	0.9002	12.0	17	6	17.2429
1.1881	18.0	144	1.4628	0.5784	0.3312	0.5121	0.5126	0.8866	0.8993	12.0429	17	6	17.3071
1.1721	19.0	152	1.4589	0.5754	0.3284	0.5105	0.5114	0.8874	0.8993	11.9571	17	6	17.2143
1.1419	20.0	160	1.4561	0.5748	0.3296	0.511	0.511	0.8873	0.8993	11.9786	17	6	17.2357
1.1299	21.0	168	1.4605	0.5813	0.3349	0.518	0.518	0.8876	0.9006	12.1357	18	6	17.35
1.1295	22.0	176	1.4605	0.5756	0.3292	0.512	0.5117	0.8874	0.8985	11.95	17	6	17.1714
1.1091	23.0	184	1.4609	0.5746	0.3277	0.5129	0.5129	0.8877	0.899	11.9571	17	6	17.1857
1.0963	24.0	192	1.4616	0.5715	0.3236	0.5101	0.5096	0.8868	0.8987	11.9571	17	6	17.25
1.0713	25.0	200	1.4590	0.5733	0.3264	0.5119	0.5117	0.8872	0.8992	11.9857	17	6	17.2286
1.0578	26.0	208	1.4569	0.577	0.3317	0.5139	0.5141	0.8888	0.8996	11.9071	17	6	17.1143
1.0416	27.0	216	1.4638	0.5761	0.3312	0.5145	0.5138	0.8883	0.8994	12.0071	18	6	17.2071
1.0398	28.0	224	1.4657	0.5784	0.3351	0.5149	0.515	0.8887	0.8992	11.9	18	6	17.0429
1.0286	29.0	232	1.4684	0.5776	0.335	0.5164	0.516	0.8889	0.8992	11.9429	18	6	17.1
1.0095	30.0	240	1.4734	0.5772	0.3381	0.5178	0.5177	0.8886	0.8989	11.9143	18	6	17.1214
1.0093	31.0	248	1.4737	0.5776	0.3374	0.5193	0.5188	0.889	0.8998	11.8714	18	6	17.1
0.9892	32.0	256	1.4707	0.5836	0.3469	0.5246	0.5251	0.8902	0.9005	11.7929	18	6	16.9786
0.9982	33.0	264	1.4734	0.5832	0.3444	0.5249	0.5248	0.89	0.9004	11.8571	18	6	17.0929
0.983	34.0	272	1.4767	0.5804	0.3427	0.5224	0.5221	0.8899	0.8997	11.7286	18	6	17.0071
0.962	35.0	280	1.4790	0.5805	0.3402	0.5215	0.5214	0.8901	0.8995	11.6929	18	6	16.9643
0.9575	36.0	288	1.4817	0.5817	0.3411	0.5209	0.5214	0.8906	0.9001	11.6143	18	6	16.8714
0.948	37.0	296	1.4842	0.5823	0.3421	0.522	0.5224	0.891	0.8999	11.6429	18	6	16.8714
0.9448	38.0	304	1.4843	0.5812	0.3426	0.5223	0.5223	0.891	0.8999	11.5786	18	6	16.8143
0.9415	39.0	312	1.4860	0.5802	0.3419	0.5203	0.52	0.8909	0.8992	11.5357	18	6	16.7786
0.9536	40.0	320	1.4868	0.5801	0.3382	0.5198	0.5195	0.8906	0.8982	11.5429	18	6	16.7286
0.9249	41.0	328	1.4891	0.5804	0.3386	0.5203	0.5201	0.8917	0.8994	11.5929	18	6	16.7857
0.9287	42.0	336	1.4904	0.5767	0.3397	0.5181	0.5181	0.8906	0.8994	11.6429	18	6	16.8929
0.94	43.0	344	1.4923	0.5824	0.3431	0.5227	0.5227	0.8918	0.9011	11.6429	18	6	16.8929
0.9118	44.0	352	1.4921	0.5835	0.3442	0.5238	0.524	0.8924	0.9013	11.6286	18	6	16.8429
0.9343	45.0	360	1.4907	0.5824	0.3438	0.5225	0.5228	0.8921	0.9011	11.6286	18	6	16.8571
0.9133	46.0	368	1.4902	0.584	0.3453	0.5236	0.5236	0.893	0.9013	11.6	18	6	16.8071
0.9162	47.0	376	1.4903	0.584	0.3453	0.5236	0.5236	0.8929	0.9012	11.5929	18	6	16.8071
0.9088	48.0	384	1.4904	0.5848	0.3454	0.5243	0.5242	0.8931	0.9013	11.6	18	6	16.8
0.9225	49.0	392	1.4908	0.5855	0.3458	0.525	0.5248	0.8932	0.9014	11.6	18	6	16.8
0.9215	50.0	400	1.4907	0.5855	0.3458	0.525	0.5248	0.8932	0.9014	11.6	18	6	16.8

Framework versions

• Transformers 4.32.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3