|
--- |
|
license: apache-2.0 |
|
base_model: t5-small |
|
tags: |
|
- generated_from_trainer |
|
metrics: |
|
- rouge |
|
model-index: |
|
- name: text_shortening_model_v13 |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# text_shortening_model_v13 |
|
|
|
This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the None dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 2.0548 |
|
- Rouge1: 0.5772 |
|
- Rouge2: 0.3353 |
|
- Rougel: 0.5189 |
|
- Rougelsum: 0.5189 |
|
- Bert precision: 0.8941 |
|
- Bert recall: 0.8987 |
|
- Average word count: 11.2143 |
|
- Max word count: 15 |
|
- Min word count: 6 |
|
- Average token count: 16.5071 |
|
- % shortened texts with length > 12: 30.0 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0001 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 60 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Bert precision | Bert recall | Average word count | Max word count | Min word count | Average token count | % shortened texts with length > 12 | |
|
|:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:--------------:|:-----------:|:------------------:|:--------------:|:--------------:|:-------------------:|:----------------------------------:| |
|
| 1.1544 | 1.0 | 62 | 1.6383 | 0.5496 | 0.3173 | 0.5011 | 0.5016 | 0.8821 | 0.8901 | 11.7214 | 17 | 4 | 17.0214 | 50.0 | |
|
| 1.0203 | 2.0 | 124 | 1.5337 | 0.569 | 0.3214 | 0.5114 | 0.5112 | 0.8863 | 0.8968 | 11.9857 | 17 | 5 | 17.2214 | 50.7143 | |
|
| 0.9474 | 3.0 | 186 | 1.5169 | 0.5754 | 0.3283 | 0.5196 | 0.5186 | 0.8874 | 0.8985 | 12.0857 | 17 | 6 | 17.2571 | 53.5714 | |
|
| 0.8615 | 4.0 | 248 | 1.5058 | 0.5785 | 0.3368 | 0.5211 | 0.5198 | 0.8917 | 0.9006 | 11.7 | 17 | 6 | 16.8714 | 47.8571 | |
|
| 0.8182 | 5.0 | 310 | 1.4855 | 0.5817 | 0.3284 | 0.5203 | 0.5195 | 0.8907 | 0.9 | 11.7357 | 17 | 6 | 16.9357 | 46.4286 | |
|
| 0.784 | 6.0 | 372 | 1.4813 | 0.5862 | 0.3398 | 0.5242 | 0.5242 | 0.8918 | 0.9016 | 11.7 | 16 | 6 | 17.0 | 45.0 | |
|
| 0.7749 | 7.0 | 434 | 1.4723 | 0.581 | 0.334 | 0.5241 | 0.5233 | 0.8951 | 0.8984 | 11.1929 | 16 | 6 | 16.3286 | 32.1429 | |
|
| 0.7396 | 8.0 | 496 | 1.4936 | 0.5791 | 0.3402 | 0.5184 | 0.5183 | 0.8933 | 0.8992 | 11.4786 | 17 | 6 | 16.5571 | 34.2857 | |
|
| 0.6856 | 9.0 | 558 | 1.5083 | 0.5757 | 0.3364 | 0.5174 | 0.5172 | 0.8944 | 0.8979 | 11.2 | 16 | 6 | 16.2357 | 30.7143 | |
|
| 0.6679 | 10.0 | 620 | 1.5295 | 0.5814 | 0.3399 | 0.5271 | 0.5276 | 0.8915 | 0.9 | 11.7786 | 16 | 7 | 16.9143 | 40.0 | |
|
| 0.6506 | 11.0 | 682 | 1.5363 | 0.5829 | 0.3491 | 0.5282 | 0.5283 | 0.8953 | 0.8994 | 11.3786 | 16 | 6 | 16.5286 | 33.5714 | |
|
| 0.6521 | 12.0 | 744 | 1.5526 | 0.5645 | 0.3303 | 0.5095 | 0.5096 | 0.8914 | 0.8951 | 11.2286 | 16 | 5 | 16.4929 | 30.7143 | |
|
| 0.6125 | 13.0 | 806 | 1.5787 | 0.5709 | 0.324 | 0.5097 | 0.5108 | 0.8906 | 0.8953 | 11.4214 | 16 | 6 | 16.6571 | 35.0 | |
|
| 0.5915 | 14.0 | 868 | 1.5946 | 0.5757 | 0.3373 | 0.5152 | 0.5159 | 0.8926 | 0.8969 | 11.4071 | 16 | 6 | 16.5571 | 32.8571 | |
|
| 0.5737 | 15.0 | 930 | 1.6204 | 0.577 | 0.3322 | 0.5219 | 0.5223 | 0.8918 | 0.8986 | 11.5929 | 16 | 6 | 16.8214 | 35.7143 | |
|
| 0.5812 | 16.0 | 992 | 1.6372 | 0.5748 | 0.3243 | 0.52 | 0.5203 | 0.891 | 0.8977 | 11.6071 | 16 | 7 | 16.8214 | 37.8571 | |
|
| 0.5468 | 17.0 | 1054 | 1.6514 | 0.5673 | 0.3304 | 0.5152 | 0.5152 | 0.895 | 0.8954 | 11.0 | 15 | 5 | 15.9929 | 26.4286 | |
|
| 0.56 | 18.0 | 1116 | 1.6630 | 0.576 | 0.3273 | 0.5229 | 0.5228 | 0.8907 | 0.898 | 11.5786 | 16 | 6 | 16.8429 | 35.0 | |
|
| 0.5548 | 19.0 | 1178 | 1.6868 | 0.5739 | 0.3262 | 0.5139 | 0.5135 | 0.8923 | 0.8972 | 11.3429 | 16 | 6 | 16.5929 | 33.5714 | |
|
| 0.5338 | 20.0 | 1240 | 1.6954 | 0.5702 | 0.3295 | 0.518 | 0.5182 | 0.8914 | 0.8975 | 11.6 | 16 | 6 | 16.7429 | 37.8571 | |
|
| 0.5323 | 21.0 | 1302 | 1.7255 | 0.585 | 0.3376 | 0.5262 | 0.5266 | 0.8938 | 0.9007 | 11.5643 | 16 | 6 | 16.7429 | 35.0 | |
|
| 0.5075 | 22.0 | 1364 | 1.7320 | 0.5708 | 0.3272 | 0.5137 | 0.5144 | 0.8929 | 0.8953 | 11.3286 | 16 | 6 | 16.4143 | 32.1429 | |
|
| 0.4916 | 23.0 | 1426 | 1.7601 | 0.5724 | 0.3276 | 0.5161 | 0.5171 | 0.8928 | 0.8965 | 11.3357 | 16 | 6 | 16.6 | 31.4286 | |
|
| 0.4789 | 24.0 | 1488 | 1.7779 | 0.5726 | 0.3253 | 0.5128 | 0.513 | 0.8934 | 0.8964 | 11.4143 | 16 | 6 | 16.5 | 35.0 | |
|
| 0.4851 | 25.0 | 1550 | 1.7970 | 0.575 | 0.3318 | 0.5204 | 0.521 | 0.8935 | 0.8982 | 11.4429 | 16 | 6 | 16.6714 | 31.4286 | |
|
| 0.4682 | 26.0 | 1612 | 1.8094 | 0.5783 | 0.3376 | 0.5203 | 0.5213 | 0.8937 | 0.8984 | 11.4714 | 16 | 6 | 16.7571 | 30.0 | |
|
| 0.4703 | 27.0 | 1674 | 1.8299 | 0.5814 | 0.3383 | 0.5208 | 0.5215 | 0.8934 | 0.8982 | 11.3929 | 16 | 6 | 16.6643 | 30.0 | |
|
| 0.483 | 28.0 | 1736 | 1.8396 | 0.576 | 0.3394 | 0.5155 | 0.5162 | 0.8945 | 0.8975 | 11.3357 | 16 | 6 | 16.3857 | 28.5714 | |
|
| 0.4712 | 29.0 | 1798 | 1.8567 | 0.5741 | 0.326 | 0.5125 | 0.5129 | 0.893 | 0.8981 | 11.5 | 16 | 6 | 16.5786 | 35.7143 | |
|
| 0.4679 | 30.0 | 1860 | 1.8818 | 0.5855 | 0.3416 | 0.5239 | 0.5242 | 0.895 | 0.9005 | 11.4429 | 16 | 6 | 16.7143 | 33.5714 | |
|
| 0.4653 | 31.0 | 1922 | 1.8758 | 0.5805 | 0.3378 | 0.5217 | 0.5222 | 0.894 | 0.8986 | 11.3357 | 16 | 6 | 16.4857 | 30.0 | |
|
| 0.4484 | 32.0 | 1984 | 1.8920 | 0.5812 | 0.3363 | 0.5207 | 0.5206 | 0.8946 | 0.8991 | 11.3357 | 16 | 6 | 16.5143 | 30.0 | |
|
| 0.4428 | 33.0 | 2046 | 1.8925 | 0.5832 | 0.3372 | 0.5195 | 0.5203 | 0.8968 | 0.8987 | 11.1286 | 16 | 6 | 16.2214 | 26.4286 | |
|
| 0.4266 | 34.0 | 2108 | 1.9185 | 0.5736 | 0.3322 | 0.517 | 0.518 | 0.8952 | 0.8974 | 11.0214 | 15 | 6 | 16.1714 | 25.0 | |
|
| 0.429 | 35.0 | 2170 | 1.9366 | 0.5829 | 0.3371 | 0.5224 | 0.5231 | 0.8965 | 0.8988 | 11.1643 | 16 | 6 | 16.25 | 27.1429 | |
|
| 0.4034 | 36.0 | 2232 | 1.9510 | 0.5823 | 0.3392 | 0.5288 | 0.5288 | 0.8963 | 0.8986 | 11.1143 | 15 | 6 | 16.1214 | 30.0 | |
|
| 0.4111 | 37.0 | 2294 | 1.9517 | 0.587 | 0.3426 | 0.529 | 0.5296 | 0.8959 | 0.9011 | 11.3857 | 16 | 6 | 16.55 | 31.4286 | |
|
| 0.4318 | 38.0 | 2356 | 1.9450 | 0.5851 | 0.3444 | 0.5262 | 0.5268 | 0.8963 | 0.9009 | 11.2714 | 16 | 6 | 16.4571 | 30.0 | |
|
| 0.4399 | 39.0 | 2418 | 1.9539 | 0.5772 | 0.3339 | 0.5164 | 0.5169 | 0.8958 | 0.8995 | 11.0929 | 15 | 6 | 16.2929 | 25.0 | |
|
| 0.4268 | 40.0 | 2480 | 1.9620 | 0.5806 | 0.3319 | 0.5187 | 0.5188 | 0.8962 | 0.8983 | 11.0214 | 16 | 6 | 16.0643 | 26.4286 | |
|
| 0.4119 | 41.0 | 2542 | 1.9939 | 0.5819 | 0.3408 | 0.5239 | 0.5238 | 0.8945 | 0.8992 | 11.3 | 16 | 6 | 16.4929 | 30.0 | |
|
| 0.4061 | 42.0 | 2604 | 1.9714 | 0.5813 | 0.338 | 0.5214 | 0.5228 | 0.897 | 0.8997 | 11.05 | 16 | 6 | 16.2429 | 25.7143 | |
|
| 0.4176 | 43.0 | 2666 | 1.9911 | 0.5847 | 0.3388 | 0.5266 | 0.5265 | 0.8951 | 0.9003 | 11.1929 | 16 | 6 | 16.4643 | 28.5714 | |
|
| 0.4041 | 44.0 | 2728 | 2.0105 | 0.5844 | 0.3468 | 0.5257 | 0.5256 | 0.8957 | 0.901 | 11.1786 | 15 | 6 | 16.5357 | 29.2857 | |
|
| 0.3925 | 45.0 | 2790 | 2.0220 | 0.5787 | 0.3423 | 0.5179 | 0.5185 | 0.8936 | 0.8992 | 11.25 | 16 | 6 | 16.5143 | 32.1429 | |
|
| 0.4095 | 46.0 | 2852 | 2.0179 | 0.581 | 0.3404 | 0.5197 | 0.5202 | 0.8957 | 0.8998 | 11.2143 | 16 | 6 | 16.4357 | 29.2857 | |
|
| 0.397 | 47.0 | 2914 | 2.0124 | 0.5803 | 0.3385 | 0.5188 | 0.5193 | 0.8952 | 0.899 | 11.2357 | 16 | 6 | 16.2786 | 32.1429 | |
|
| 0.3801 | 48.0 | 2976 | 2.0186 | 0.5778 | 0.3359 | 0.518 | 0.518 | 0.8944 | 0.8986 | 11.2143 | 16 | 6 | 16.4 | 32.1429 | |
|
| 0.3966 | 49.0 | 3038 | 2.0234 | 0.5807 | 0.337 | 0.5185 | 0.5192 | 0.8953 | 0.9001 | 11.2571 | 16 | 6 | 16.4929 | 30.0 | |
|
| 0.3838 | 50.0 | 3100 | 2.0317 | 0.5807 | 0.3427 | 0.523 | 0.5234 | 0.8954 | 0.8989 | 11.0571 | 16 | 6 | 16.2786 | 26.4286 | |
|
| 0.3818 | 51.0 | 3162 | 2.0281 | 0.5811 | 0.3428 | 0.5238 | 0.5242 | 0.8956 | 0.9001 | 11.1643 | 16 | 6 | 16.3643 | 30.7143 | |
|
| 0.3793 | 52.0 | 3224 | 2.0399 | 0.5824 | 0.3438 | 0.5214 | 0.522 | 0.8947 | 0.9003 | 11.2071 | 16 | 6 | 16.4714 | 30.7143 | |
|
| 0.3734 | 53.0 | 3286 | 2.0470 | 0.5811 | 0.3413 | 0.5222 | 0.5227 | 0.8952 | 0.9 | 11.1643 | 15 | 6 | 16.4214 | 29.2857 | |
|
| 0.3876 | 54.0 | 3348 | 2.0509 | 0.5764 | 0.3382 | 0.515 | 0.5156 | 0.8948 | 0.8983 | 11.1071 | 15 | 6 | 16.2643 | 28.5714 | |
|
| 0.3736 | 55.0 | 3410 | 2.0498 | 0.5722 | 0.3331 | 0.5135 | 0.514 | 0.8937 | 0.8972 | 11.1357 | 16 | 6 | 16.3071 | 27.8571 | |
|
| 0.3981 | 56.0 | 3472 | 2.0499 | 0.5726 | 0.3337 | 0.5133 | 0.5138 | 0.8939 | 0.8977 | 11.1286 | 15 | 6 | 16.3357 | 29.2857 | |
|
| 0.3731 | 57.0 | 3534 | 2.0500 | 0.5767 | 0.3353 | 0.5173 | 0.5176 | 0.8946 | 0.8984 | 11.1286 | 15 | 6 | 16.3643 | 27.8571 | |
|
| 0.3786 | 58.0 | 3596 | 2.0529 | 0.5779 | 0.3377 | 0.5199 | 0.5208 | 0.895 | 0.8994 | 11.1929 | 16 | 6 | 16.4357 | 28.5714 | |
|
| 0.3648 | 59.0 | 3658 | 2.0545 | 0.5766 | 0.3348 | 0.518 | 0.5181 | 0.8939 | 0.8985 | 11.2143 | 15 | 6 | 16.5143 | 30.0 | |
|
| 0.373 | 60.0 | 3720 | 2.0548 | 0.5772 | 0.3353 | 0.5189 | 0.5189 | 0.8941 | 0.8987 | 11.2143 | 15 | 6 | 16.5071 | 30.0 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.33.1 |
|
- Pytorch 2.0.1+cpu |
|
- Datasets 2.14.5 |
|
- Tokenizers 0.13.3 |
|
|
