text_shortening_model_v59

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8420
• Rouge1: 0.7121
• Rouge2: 0.5315
• Rougel: 0.6662
• Rougelsum: 0.6654
• Bert precision: 0.9198
• Bert recall: 0.9174
• Bert f1-score: 0.9181
• Average word count: 8.2143
• Max word count: 14
• Min word count: 3
• Average token count: 12.6027
• % shortened texts with length > 12: 3.5714

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bert precision	Bert recall	Bert f1-score	Average word count	Max word count	Min word count	Average token count	% shortened texts with length > 12
1.726	1.0	49	0.9545	0.6393	0.4643	0.5917	0.5904	0.915	0.9008	0.9074	7.7188	16	2	11.8973	6.25
1.1192	2.0	98	0.8062	0.6679	0.4944	0.6206	0.6198	0.9197	0.9081	0.9134	7.7679	15	2	11.9464	4.9107
0.9313	3.0	147	0.7383	0.69	0.5136	0.6436	0.6429	0.9229	0.9139	0.918	7.8616	16	3	12.1607	4.4643
0.8361	4.0	196	0.6998	0.6935	0.5096	0.655	0.6541	0.9218	0.9147	0.9178	8.0134	16	3	12.317	4.4643
0.7703	5.0	245	0.6773	0.6956	0.5188	0.6599	0.6594	0.9212	0.914	0.9171	8.0268	16	3	12.2812	5.8036
0.7075	6.0	294	0.6767	0.7084	0.5298	0.6686	0.6671	0.9256	0.9167	0.9207	7.9241	16	3	12.2054	4.9107
0.6389	7.0	343	0.6764	0.6975	0.5105	0.6541	0.6538	0.9208	0.9149	0.9173	8.0759	16	3	12.4241	5.3571
0.5976	8.0	392	0.6668	0.7066	0.5211	0.669	0.6673	0.9239	0.9146	0.9187	7.7902	16	3	12.1562	3.5714
0.5648	9.0	441	0.6656	0.6958	0.5109	0.6555	0.6541	0.9207	0.9129	0.9163	7.8929	16	3	12.3482	3.5714
0.5495	10.0	490	0.6754	0.7034	0.5113	0.6594	0.659	0.9245	0.9136	0.9186	7.8705	15	3	12.1786	2.6786
0.5188	11.0	539	0.6730	0.6899	0.496	0.646	0.6462	0.9211	0.9094	0.9148	7.7634	15	3	12.183	3.5714
0.4838	12.0	588	0.6929	0.7031	0.5212	0.6576	0.656	0.9232	0.9123	0.9173	7.6964	15	3	12.0491	2.6786
0.4639	13.0	637	0.6917	0.6997	0.5107	0.6512	0.651	0.9196	0.9123	0.9154	7.9732	15	3	12.2902	3.125
0.437	14.0	686	0.7137	0.6965	0.5111	0.6562	0.6553	0.9183	0.9142	0.9158	8.1339	15	3	12.5536	4.9107
0.4348	15.0	735	0.7032	0.697	0.4967	0.6442	0.6429	0.9194	0.9107	0.9146	7.9375	15	3	12.2723	2.2321
0.4134	16.0	784	0.7143	0.7059	0.5124	0.6531	0.6522	0.9207	0.9138	0.9168	7.9554	15	3	12.3393	4.9107
0.4017	17.0	833	0.7179	0.7025	0.5092	0.6541	0.6529	0.92	0.9129	0.916	7.9241	16	3	12.2723	3.125
0.3789	18.0	882	0.7289	0.6973	0.509	0.6468	0.6456	0.9201	0.9127	0.9159	7.9509	15	3	12.3259	3.5714
0.3777	19.0	931	0.7276	0.702	0.5112	0.6485	0.6483	0.9195	0.9133	0.9159	7.9732	16	3	12.3125	3.5714
0.3619	20.0	980	0.7388	0.7009	0.5109	0.6483	0.6472	0.9185	0.9126	0.9151	8.0848	16	3	12.3438	2.6786
0.3494	21.0	1029	0.7469	0.7035	0.5129	0.6506	0.6499	0.918	0.9141	0.9155	8.2902	16	3	12.6295	4.0179
0.3391	22.0	1078	0.7510	0.6934	0.5032	0.6425	0.6417	0.9156	0.9122	0.9135	8.1473	15	3	12.5268	3.125
0.3163	23.0	1127	0.7658	0.6952	0.5072	0.6443	0.6432	0.9177	0.9134	0.9151	8.1875	15	3	12.5268	4.0179
0.3138	24.0	1176	0.7743	0.6901	0.4992	0.6389	0.6385	0.918	0.9105	0.9138	7.9732	15	3	12.2634	2.6786
0.3185	25.0	1225	0.7561	0.7039	0.523	0.6595	0.6601	0.9198	0.9143	0.9165	8.1429	15	3	12.4911	3.125
0.3019	26.0	1274	0.7693	0.6949	0.5044	0.6405	0.6398	0.9179	0.9122	0.9145	8.0893	15	3	12.4688	4.0179
0.2885	27.0	1323	0.7774	0.6991	0.5072	0.6436	0.6425	0.9193	0.9129	0.9156	8.058	14	3	12.433	2.6786
0.2922	28.0	1372	0.7932	0.7038	0.5199	0.6601	0.6599	0.9169	0.915	0.9154	8.2902	15	3	12.7277	4.0179
0.2794	29.0	1421	0.7921	0.7123	0.5251	0.6654	0.6643	0.9215	0.9177	0.9191	8.2098	15	3	12.6027	4.9107
0.2756	30.0	1470	0.7889	0.7072	0.5217	0.6582	0.6577	0.92	0.9144	0.9167	8.0759	15	3	12.4107	2.6786
0.2658	31.0	1519	0.7950	0.7037	0.5197	0.6523	0.6515	0.9192	0.9139	0.916	8.0759	15	3	12.4598	3.125
0.2722	32.0	1568	0.7974	0.7089	0.5267	0.663	0.6628	0.9218	0.915	0.9179	7.9688	14	3	12.2455	2.6786
0.2575	33.0	1617	0.7979	0.7052	0.5199	0.6569	0.6558	0.9186	0.9151	0.9164	8.1116	15	3	12.4955	3.5714
0.2544	34.0	1666	0.7992	0.7082	0.53	0.6608	0.6599	0.9203	0.9168	0.9181	8.1741	14	3	12.567	4.0179
0.2572	35.0	1715	0.8005	0.7123	0.5286	0.664	0.6624	0.922	0.9168	0.919	8.0714	14	3	12.4196	3.5714
0.2455	36.0	1764	0.8001	0.7104	0.5271	0.6634	0.6639	0.9204	0.9166	0.918	8.1295	14	3	12.5268	3.5714
0.2434	37.0	1813	0.8072	0.7112	0.5276	0.6645	0.6632	0.9201	0.9166	0.9178	8.1607	14	3	12.5491	4.0179
0.2375	38.0	1862	0.8120	0.7079	0.5252	0.663	0.6624	0.9199	0.9159	0.9174	8.1741	14	3	12.5357	3.5714
0.2271	39.0	1911	0.8156	0.7017	0.5193	0.655	0.6547	0.9166	0.9153	0.9155	8.2723	14	3	12.7009	3.5714
0.2349	40.0	1960	0.8194	0.7068	0.5246	0.6594	0.6592	0.9209	0.9161	0.9181	8.0982	14	3	12.4464	3.5714
0.2262	41.0	2009	0.8266	0.7107	0.5287	0.6641	0.6634	0.9206	0.9177	0.9187	8.1652	14	3	12.4777	3.5714
0.2154	42.0	2058	0.8313	0.7094	0.5286	0.6636	0.6627	0.9208	0.9176	0.9187	8.1562	14	3	12.5268	3.125
0.2274	43.0	2107	0.8342	0.7101	0.5296	0.6644	0.6639	0.9214	0.9179	0.9192	8.183	14	3	12.5402	3.125
0.2229	44.0	2156	0.8378	0.7077	0.5278	0.6622	0.6616	0.9198	0.9166	0.9178	8.1518	14	3	12.5714	3.125
0.2263	45.0	2205	0.8417	0.7087	0.5293	0.6633	0.6631	0.9197	0.9165	0.9176	8.1652	14	3	12.567	3.125
0.2255	46.0	2254	0.8417	0.7075	0.5248	0.6613	0.6615	0.9198	0.9163	0.9176	8.125	14	3	12.4821	3.125
0.2195	47.0	2303	0.8415	0.708	0.5299	0.6642	0.664	0.9196	0.9167	0.9176	8.183	14	3	12.5804	3.125
0.2036	48.0	2352	0.8412	0.7076	0.5271	0.6626	0.6622	0.9195	0.9166	0.9176	8.2054	14	3	12.6071	3.5714
0.2208	49.0	2401	0.8416	0.7114	0.5306	0.666	0.6653	0.9201	0.9172	0.9182	8.2054	14	3	12.5893	3.5714
0.2088	50.0	2450	0.8420	0.7121	0.5315	0.6662	0.6654	0.9198	0.9174	0.9181	8.2143	14	3	12.6027	3.5714

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3