<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
text_shortening_model_v59
This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.8420
- Rouge1: 0.7121
- Rouge2: 0.5315
- Rougel: 0.6662
- Rougelsum: 0.6654
- Bert precision: 0.9198
- Bert recall: 0.9174
- Bert f1-score: 0.9181
- Average word count: 8.2143
- Max word count: 14
- Min word count: 3
- Average token count: 12.6027
- % shortened texts with length > 12: 3.5714
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50
Training results
Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Bert precision | Bert recall | Bert f1-score | Average word count | Max word count | Min word count | Average token count | % shortened texts with length > 12 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1.726 | 1.0 | 49 | 0.9545 | 0.6393 | 0.4643 | 0.5917 | 0.5904 | 0.915 | 0.9008 | 0.9074 | 7.7188 | 16 | 2 | 11.8973 | 6.25 |
1.1192 | 2.0 | 98 | 0.8062 | 0.6679 | 0.4944 | 0.6206 | 0.6198 | 0.9197 | 0.9081 | 0.9134 | 7.7679 | 15 | 2 | 11.9464 | 4.9107 |
0.9313 | 3.0 | 147 | 0.7383 | 0.69 | 0.5136 | 0.6436 | 0.6429 | 0.9229 | 0.9139 | 0.918 | 7.8616 | 16 | 3 | 12.1607 | 4.4643 |
0.8361 | 4.0 | 196 | 0.6998 | 0.6935 | 0.5096 | 0.655 | 0.6541 | 0.9218 | 0.9147 | 0.9178 | 8.0134 | 16 | 3 | 12.317 | 4.4643 |
0.7703 | 5.0 | 245 | 0.6773 | 0.6956 | 0.5188 | 0.6599 | 0.6594 | 0.9212 | 0.914 | 0.9171 | 8.0268 | 16 | 3 | 12.2812 | 5.8036 |
0.7075 | 6.0 | 294 | 0.6767 | 0.7084 | 0.5298 | 0.6686 | 0.6671 | 0.9256 | 0.9167 | 0.9207 | 7.9241 | 16 | 3 | 12.2054 | 4.9107 |
0.6389 | 7.0 | 343 | 0.6764 | 0.6975 | 0.5105 | 0.6541 | 0.6538 | 0.9208 | 0.9149 | 0.9173 | 8.0759 | 16 | 3 | 12.4241 | 5.3571 |
0.5976 | 8.0 | 392 | 0.6668 | 0.7066 | 0.5211 | 0.669 | 0.6673 | 0.9239 | 0.9146 | 0.9187 | 7.7902 | 16 | 3 | 12.1562 | 3.5714 |
0.5648 | 9.0 | 441 | 0.6656 | 0.6958 | 0.5109 | 0.6555 | 0.6541 | 0.9207 | 0.9129 | 0.9163 | 7.8929 | 16 | 3 | 12.3482 | 3.5714 |
0.5495 | 10.0 | 490 | 0.6754 | 0.7034 | 0.5113 | 0.6594 | 0.659 | 0.9245 | 0.9136 | 0.9186 | 7.8705 | 15 | 3 | 12.1786 | 2.6786 |
0.5188 | 11.0 | 539 | 0.6730 | 0.6899 | 0.496 | 0.646 | 0.6462 | 0.9211 | 0.9094 | 0.9148 | 7.7634 | 15 | 3 | 12.183 | 3.5714 |
0.4838 | 12.0 | 588 | 0.6929 | 0.7031 | 0.5212 | 0.6576 | 0.656 | 0.9232 | 0.9123 | 0.9173 | 7.6964 | 15 | 3 | 12.0491 | 2.6786 |
0.4639 | 13.0 | 637 | 0.6917 | 0.6997 | 0.5107 | 0.6512 | 0.651 | 0.9196 | 0.9123 | 0.9154 | 7.9732 | 15 | 3 | 12.2902 | 3.125 |
0.437 | 14.0 | 686 | 0.7137 | 0.6965 | 0.5111 | 0.6562 | 0.6553 | 0.9183 | 0.9142 | 0.9158 | 8.1339 | 15 | 3 | 12.5536 | 4.9107 |
0.4348 | 15.0 | 735 | 0.7032 | 0.697 | 0.4967 | 0.6442 | 0.6429 | 0.9194 | 0.9107 | 0.9146 | 7.9375 | 15 | 3 | 12.2723 | 2.2321 |
0.4134 | 16.0 | 784 | 0.7143 | 0.7059 | 0.5124 | 0.6531 | 0.6522 | 0.9207 | 0.9138 | 0.9168 | 7.9554 | 15 | 3 | 12.3393 | 4.9107 |
0.4017 | 17.0 | 833 | 0.7179 | 0.7025 | 0.5092 | 0.6541 | 0.6529 | 0.92 | 0.9129 | 0.916 | 7.9241 | 16 | 3 | 12.2723 | 3.125 |
0.3789 | 18.0 | 882 | 0.7289 | 0.6973 | 0.509 | 0.6468 | 0.6456 | 0.9201 | 0.9127 | 0.9159 | 7.9509 | 15 | 3 | 12.3259 | 3.5714 |
0.3777 | 19.0 | 931 | 0.7276 | 0.702 | 0.5112 | 0.6485 | 0.6483 | 0.9195 | 0.9133 | 0.9159 | 7.9732 | 16 | 3 | 12.3125 | 3.5714 |
0.3619 | 20.0 | 980 | 0.7388 | 0.7009 | 0.5109 | 0.6483 | 0.6472 | 0.9185 | 0.9126 | 0.9151 | 8.0848 | 16 | 3 | 12.3438 | 2.6786 |
0.3494 | 21.0 | 1029 | 0.7469 | 0.7035 | 0.5129 | 0.6506 | 0.6499 | 0.918 | 0.9141 | 0.9155 | 8.2902 | 16 | 3 | 12.6295 | 4.0179 |
0.3391 | 22.0 | 1078 | 0.7510 | 0.6934 | 0.5032 | 0.6425 | 0.6417 | 0.9156 | 0.9122 | 0.9135 | 8.1473 | 15 | 3 | 12.5268 | 3.125 |
0.3163 | 23.0 | 1127 | 0.7658 | 0.6952 | 0.5072 | 0.6443 | 0.6432 | 0.9177 | 0.9134 | 0.9151 | 8.1875 | 15 | 3 | 12.5268 | 4.0179 |
0.3138 | 24.0 | 1176 | 0.7743 | 0.6901 | 0.4992 | 0.6389 | 0.6385 | 0.918 | 0.9105 | 0.9138 | 7.9732 | 15 | 3 | 12.2634 | 2.6786 |
0.3185 | 25.0 | 1225 | 0.7561 | 0.7039 | 0.523 | 0.6595 | 0.6601 | 0.9198 | 0.9143 | 0.9165 | 8.1429 | 15 | 3 | 12.4911 | 3.125 |
0.3019 | 26.0 | 1274 | 0.7693 | 0.6949 | 0.5044 | 0.6405 | 0.6398 | 0.9179 | 0.9122 | 0.9145 | 8.0893 | 15 | 3 | 12.4688 | 4.0179 |
0.2885 | 27.0 | 1323 | 0.7774 | 0.6991 | 0.5072 | 0.6436 | 0.6425 | 0.9193 | 0.9129 | 0.9156 | 8.058 | 14 | 3 | 12.433 | 2.6786 |
0.2922 | 28.0 | 1372 | 0.7932 | 0.7038 | 0.5199 | 0.6601 | 0.6599 | 0.9169 | 0.915 | 0.9154 | 8.2902 | 15 | 3 | 12.7277 | 4.0179 |
0.2794 | 29.0 | 1421 | 0.7921 | 0.7123 | 0.5251 | 0.6654 | 0.6643 | 0.9215 | 0.9177 | 0.9191 | 8.2098 | 15 | 3 | 12.6027 | 4.9107 |
0.2756 | 30.0 | 1470 | 0.7889 | 0.7072 | 0.5217 | 0.6582 | 0.6577 | 0.92 | 0.9144 | 0.9167 | 8.0759 | 15 | 3 | 12.4107 | 2.6786 |
0.2658 | 31.0 | 1519 | 0.7950 | 0.7037 | 0.5197 | 0.6523 | 0.6515 | 0.9192 | 0.9139 | 0.916 | 8.0759 | 15 | 3 | 12.4598 | 3.125 |
0.2722 | 32.0 | 1568 | 0.7974 | 0.7089 | 0.5267 | 0.663 | 0.6628 | 0.9218 | 0.915 | 0.9179 | 7.9688 | 14 | 3 | 12.2455 | 2.6786 |
0.2575 | 33.0 | 1617 | 0.7979 | 0.7052 | 0.5199 | 0.6569 | 0.6558 | 0.9186 | 0.9151 | 0.9164 | 8.1116 | 15 | 3 | 12.4955 | 3.5714 |
0.2544 | 34.0 | 1666 | 0.7992 | 0.7082 | 0.53 | 0.6608 | 0.6599 | 0.9203 | 0.9168 | 0.9181 | 8.1741 | 14 | 3 | 12.567 | 4.0179 |
0.2572 | 35.0 | 1715 | 0.8005 | 0.7123 | 0.5286 | 0.664 | 0.6624 | 0.922 | 0.9168 | 0.919 | 8.0714 | 14 | 3 | 12.4196 | 3.5714 |
0.2455 | 36.0 | 1764 | 0.8001 | 0.7104 | 0.5271 | 0.6634 | 0.6639 | 0.9204 | 0.9166 | 0.918 | 8.1295 | 14 | 3 | 12.5268 | 3.5714 |
0.2434 | 37.0 | 1813 | 0.8072 | 0.7112 | 0.5276 | 0.6645 | 0.6632 | 0.9201 | 0.9166 | 0.9178 | 8.1607 | 14 | 3 | 12.5491 | 4.0179 |
0.2375 | 38.0 | 1862 | 0.8120 | 0.7079 | 0.5252 | 0.663 | 0.6624 | 0.9199 | 0.9159 | 0.9174 | 8.1741 | 14 | 3 | 12.5357 | 3.5714 |
0.2271 | 39.0 | 1911 | 0.8156 | 0.7017 | 0.5193 | 0.655 | 0.6547 | 0.9166 | 0.9153 | 0.9155 | 8.2723 | 14 | 3 | 12.7009 | 3.5714 |
0.2349 | 40.0 | 1960 | 0.8194 | 0.7068 | 0.5246 | 0.6594 | 0.6592 | 0.9209 | 0.9161 | 0.9181 | 8.0982 | 14 | 3 | 12.4464 | 3.5714 |
0.2262 | 41.0 | 2009 | 0.8266 | 0.7107 | 0.5287 | 0.6641 | 0.6634 | 0.9206 | 0.9177 | 0.9187 | 8.1652 | 14 | 3 | 12.4777 | 3.5714 |
0.2154 | 42.0 | 2058 | 0.8313 | 0.7094 | 0.5286 | 0.6636 | 0.6627 | 0.9208 | 0.9176 | 0.9187 | 8.1562 | 14 | 3 | 12.5268 | 3.125 |
0.2274 | 43.0 | 2107 | 0.8342 | 0.7101 | 0.5296 | 0.6644 | 0.6639 | 0.9214 | 0.9179 | 0.9192 | 8.183 | 14 | 3 | 12.5402 | 3.125 |
0.2229 | 44.0 | 2156 | 0.8378 | 0.7077 | 0.5278 | 0.6622 | 0.6616 | 0.9198 | 0.9166 | 0.9178 | 8.1518 | 14 | 3 | 12.5714 | 3.125 |
0.2263 | 45.0 | 2205 | 0.8417 | 0.7087 | 0.5293 | 0.6633 | 0.6631 | 0.9197 | 0.9165 | 0.9176 | 8.1652 | 14 | 3 | 12.567 | 3.125 |
0.2255 | 46.0 | 2254 | 0.8417 | 0.7075 | 0.5248 | 0.6613 | 0.6615 | 0.9198 | 0.9163 | 0.9176 | 8.125 | 14 | 3 | 12.4821 | 3.125 |
0.2195 | 47.0 | 2303 | 0.8415 | 0.708 | 0.5299 | 0.6642 | 0.664 | 0.9196 | 0.9167 | 0.9176 | 8.183 | 14 | 3 | 12.5804 | 3.125 |
0.2036 | 48.0 | 2352 | 0.8412 | 0.7076 | 0.5271 | 0.6626 | 0.6622 | 0.9195 | 0.9166 | 0.9176 | 8.2054 | 14 | 3 | 12.6071 | 3.5714 |
0.2208 | 49.0 | 2401 | 0.8416 | 0.7114 | 0.5306 | 0.666 | 0.6653 | 0.9201 | 0.9172 | 0.9182 | 8.2054 | 14 | 3 | 12.5893 | 3.5714 |
0.2088 | 50.0 | 2450 | 0.8420 | 0.7121 | 0.5315 | 0.6662 | 0.6654 | 0.9198 | 0.9174 | 0.9181 | 8.2143 | 14 | 3 | 12.6027 | 3.5714 |
Framework versions
- Transformers 4.33.1
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.13.3