text_shortening_model_v12

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.8106
• Rouge1: 0.5884
• Rouge2: 0.3622
• Rougel: 0.5479
• Rougelsum: 0.5481
• Bert precision: 0.8963
• Bert recall: 0.9008
• Average word count: 11.0857
• Max word count: 16
• Min word count: 6
• Average token count: 16.1643
• % shortened texts with length > 12: 22.8571

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
1.9958	1.0	62	1.5731	0.5546	0.3304	0.5049	0.5049	0.8848	0.8911	11.3714	18	4	16.0929	42.1429
1.6826	2.0	124	1.4736	0.5884	0.3462	0.534	0.5335	0.8897	0.8994	11.6	17	4	16.8786	37.8571
1.4894	3.0	186	1.4232	0.5932	0.3573	0.5429	0.5426	0.8912	0.9016	11.7	16	6	16.9857	38.5714
1.3728	4.0	248	1.3991	0.5857	0.3546	0.5333	0.5331	0.8892	0.9008	11.6214	16	7	17.0857	37.8571
1.2891	5.0	310	1.3848	0.5982	0.3681	0.5455	0.5448	0.8913	0.9029	11.8143	17	6	17.0929	39.2857
1.1905	6.0	372	1.3698	0.5937	0.3636	0.5446	0.5437	0.8914	0.9031	11.6286	16	6	17.1	33.5714
1.1375	7.0	434	1.3690	0.5884	0.3589	0.5419	0.5413	0.8889	0.8999	11.6857	16	7	17.1071	34.2857
1.0682	8.0	496	1.3920	0.5942	0.3724	0.5499	0.5494	0.8933	0.9013	11.3786	15	6	16.5786	28.5714
1.0096	9.0	558	1.3899	0.602	0.3725	0.555	0.554	0.8954	0.9027	11.3357	16	6	16.5929	26.4286
0.9682	10.0	620	1.3983	0.6019	0.3729	0.5549	0.5542	0.8956	0.9043	11.5214	16	6	16.8357	30.0
0.9141	11.0	682	1.4009	0.5888	0.3584	0.5375	0.5364	0.892	0.9002	11.3929	16	6	16.7929	26.4286
0.8691	12.0	744	1.4216	0.5937	0.3629	0.5437	0.5434	0.8935	0.9019	11.3286	16	6	16.5143	25.7143
0.8248	13.0	806	1.4202	0.5925	0.3686	0.544	0.5443	0.8949	0.9023	11.2214	16	6	16.4929	25.0
0.7947	14.0	868	1.4393	0.5912	0.3611	0.5398	0.5387	0.8921	0.9014	11.4	16	7	16.7071	26.4286
0.7606	15.0	930	1.4555	0.5926	0.3651	0.5394	0.5386	0.8925	0.9014	11.3214	16	7	16.7714	25.7143
0.7253	16.0	992	1.4538	0.603	0.3782	0.5547	0.5538	0.8971	0.9038	11.2	16	6	16.4714	22.1429
0.7005	17.0	1054	1.4706	0.5936	0.3716	0.5471	0.5466	0.8944	0.9015	11.2143	16	6	16.5214	22.1429
0.6778	18.0	1116	1.4916	0.607	0.3818	0.5636	0.5623	0.8997	0.9048	11.0071	16	6	16.4143	19.2857
0.641	19.0	1178	1.5157	0.609	0.3817	0.5637	0.563	0.8987	0.9055	11.2214	15	6	16.4643	25.0
0.6203	20.0	1240	1.5049	0.603	0.3727	0.5517	0.5509	0.8963	0.9047	11.2857	15	6	16.7143	26.4286
0.5995	21.0	1302	1.5276	0.6033	0.3742	0.5506	0.5495	0.8966	0.9047	11.2643	15	6	16.5429	25.7143
0.5819	22.0	1364	1.5465	0.6029	0.3779	0.5568	0.5554	0.8989	0.9062	11.2929	16	6	16.6214	20.7143
0.5807	23.0	1426	1.5633	0.6008	0.3675	0.5536	0.553	0.8964	0.9039	11.2643	16	6	16.6429	24.2857
0.5374	24.0	1488	1.5787	0.606	0.379	0.5609	0.5601	0.8977	0.9065	11.4286	17	7	16.8071	23.5714
0.5226	25.0	1550	1.5796	0.6006	0.3709	0.5523	0.5514	0.8978	0.9047	11.2143	16	7	16.5357	18.5714
0.5189	26.0	1612	1.5931	0.5944	0.3623	0.5471	0.5456	0.8966	0.9022	11.1857	16	7	16.3786	21.4286
0.4965	27.0	1674	1.6016	0.6037	0.3829	0.5587	0.558	0.8993	0.9043	11.2571	16	6	16.3714	23.5714
0.5067	28.0	1736	1.6309	0.6088	0.3843	0.5671	0.5661	0.8995	0.906	11.2571	16	6	16.5357	25.7143
0.4665	29.0	1798	1.6513	0.5965	0.3733	0.5505	0.5495	0.8976	0.9024	11.2429	15	6	16.4571	22.8571
0.4676	30.0	1860	1.6501	0.6043	0.3762	0.5561	0.5555	0.8978	0.9038	11.3214	15	6	16.5857	24.2857
0.4442	31.0	1922	1.6760	0.5982	0.3784	0.559	0.5583	0.8979	0.903	11.1857	15	6	16.3786	20.7143
0.442	32.0	1984	1.6750	0.6029	0.377	0.5588	0.5583	0.898	0.9044	11.2429	15	6	16.5429	22.8571
0.4247	33.0	2046	1.6713	0.5995	0.3831	0.5563	0.5554	0.8963	0.9051	11.4857	16	7	16.7143	27.8571
0.4279	34.0	2108	1.6745	0.6047	0.3805	0.5601	0.5594	0.899	0.9052	11.2786	15	6	16.3071	27.1429
0.4116	35.0	2170	1.6822	0.6046	0.3794	0.5581	0.5579	0.8983	0.9052	11.2929	15	6	16.5071	25.7143
0.4135	36.0	2232	1.6853	0.6084	0.3875	0.564	0.564	0.8998	0.9054	11.2143	15	6	16.25	26.4286
0.392	37.0	2294	1.7124	0.6052	0.3806	0.5647	0.5646	0.8982	0.9053	11.3143	15	5	16.4786	26.4286
0.3783	38.0	2356	1.7180	0.5995	0.3784	0.5565	0.5564	0.9001	0.9035	11.0786	16	5	16.1786	22.1429
0.368	39.0	2418	1.7344	0.603	0.3869	0.5669	0.5663	0.9009	0.9047	11.0357	16	6	16.1	18.5714
0.3745	40.0	2480	1.7331	0.5961	0.3777	0.5546	0.5541	0.897	0.9024	11.2071	16	7	16.4071	20.0
0.3725	41.0	2542	1.7435	0.601	0.3785	0.557	0.557	0.898	0.9042	11.2429	16	6	16.3429	22.8571
0.3655	42.0	2604	1.7584	0.5949	0.3774	0.5533	0.5533	0.8964	0.9027	11.2143	16	7	16.35	23.5714
0.342	43.0	2666	1.7589	0.5957	0.3773	0.5538	0.5536	0.8987	0.9033	11.0571	16	7	16.0714	20.0
0.3476	44.0	2728	1.7631	0.5937	0.3761	0.5523	0.5521	0.897	0.9024	11.1643	16	7	16.3071	22.8571
0.3327	45.0	2790	1.7705	0.5903	0.3676	0.5492	0.5487	0.8966	0.902	11.1714	16	7	16.2714	23.5714
0.3412	46.0	2852	1.7684	0.5962	0.368	0.5535	0.5528	0.8968	0.9036	11.2429	16	7	16.4143	25.0
0.3376	47.0	2914	1.7731	0.5961	0.3698	0.5532	0.553	0.8968	0.9033	11.1857	15	7	16.3714	24.2857
0.3191	48.0	2976	1.7689	0.5965	0.3729	0.5518	0.5514	0.8974	0.9037	11.2071	17	6	16.35	24.2857
0.3289	49.0	3038	1.7739	0.5954	0.3711	0.5561	0.5556	0.8975	0.903	11.15	17	6	16.3214	23.5714
0.3195	50.0	3100	1.7801	0.5906	0.3637	0.551	0.5506	0.8966	0.9021	11.1286	15	7	16.2929	21.4286
0.3141	51.0	3162	1.7849	0.5977	0.3712	0.5569	0.5564	0.8985	0.9036	11.1214	16	6	16.2643	22.1429
0.3118	52.0	3224	1.7877	0.5995	0.3744	0.555	0.5547	0.8986	0.9043	11.1571	16	7	16.3286	22.1429
0.3089	53.0	3286	1.7922	0.5988	0.3687	0.5517	0.5517	0.8978	0.9036	11.15	16	6	16.2929	22.8571
0.3007	54.0	3348	1.7956	0.5948	0.3653	0.5497	0.5495	0.8988	0.9021	10.9571	16	6	16.0143	20.0
0.3063	55.0	3410	1.7993	0.5922	0.3639	0.5482	0.5473	0.8966	0.9024	11.1	16	7	16.25	22.1429
0.3074	56.0	3472	1.8007	0.5915	0.3669	0.5491	0.549	0.8969	0.9012	11.0214	16	6	16.1	21.4286
0.2941	57.0	3534	1.8052	0.5899	0.3649	0.5479	0.548	0.897	0.9009	10.9786	16	6	16.0643	20.7143
0.2923	58.0	3596	1.8096	0.5894	0.364	0.5493	0.549	0.8966	0.9015	11.0857	16	6	16.1857	22.8571
0.3076	59.0	3658	1.8105	0.5895	0.3641	0.5493	0.5491	0.8965	0.9015	11.1	16	6	16.2143	23.5714
0.3002	60.0	3720	1.8106	0.5884	0.3622	0.5479	0.5481	0.8963	0.9008	11.0857	16	6	16.1643	22.8571

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3