<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
text_shortening_model_v4
This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.4263
- Rouge1: 0.587
- Rouge2: 0.3563
- Rougel: 0.5367
- Rougelsum: 0.5356
- Bert precision: 0.8882
- Bert recall: 0.9005
- Average word count: 11.8286
- Max word count: 18
- Min word count: 6
- Average token count: 17.0929
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Bert precision | Bert recall | Average word count | Max word count | Min word count | Average token count |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1.3135 | 1.0 | 8 | 1.8236 | 0.5468 | 0.3281 | 0.4997 | 0.4987 | 0.8808 | 0.886 | 11.5786 | 18 | 6 | 16.8286 |
| 1.1741 | 2.0 | 16 | 1.6858 | 0.5482 | 0.3138 | 0.4936 | 0.4934 | 0.8776 | 0.8889 | 12.1429 | 18 | 5 | 17.2929 |
| 1.1284 | 3.0 | 24 | 1.6250 | 0.5601 | 0.3292 | 0.5053 | 0.5053 | 0.8817 | 0.8922 | 12.0357 | 18 | 5 | 17.0786 |
| 1.1142 | 4.0 | 32 | 1.5850 | 0.5645 | 0.3397 | 0.5164 | 0.516 | 0.8839 | 0.8954 | 11.9357 | 18 | 4 | 17.0571 |
| 1.0745 | 5.0 | 40 | 1.5500 | 0.5777 | 0.3465 | 0.5272 | 0.5263 | 0.8863 | 0.8995 | 12.1071 | 18 | 4 | 17.2143 |
| 1.0354 | 6.0 | 48 | 1.5235 | 0.5796 | 0.3451 | 0.5263 | 0.5252 | 0.8859 | 0.8992 | 12.0 | 18 | 5 | 17.1 |
| 1.0126 | 7.0 | 56 | 1.5026 | 0.5859 | 0.3509 | 0.53 | 0.5291 | 0.8873 | 0.8998 | 11.8786 | 18 | 5 | 17.0714 |
| 1.0087 | 8.0 | 64 | 1.4877 | 0.5828 | 0.3511 | 0.5323 | 0.5304 | 0.8869 | 0.8989 | 11.8143 | 18 | 6 | 16.9857 |
| 0.9745 | 9.0 | 72 | 1.4758 | 0.5879 | 0.3533 | 0.5343 | 0.5332 | 0.8874 | 0.9008 | 11.8857 | 18 | 6 | 17.0786 |
| 0.9712 | 10.0 | 80 | 1.4638 | 0.585 | 0.3532 | 0.5319 | 0.5303 | 0.8878 | 0.9007 | 11.8643 | 18 | 6 | 17.0643 |
| 0.9556 | 11.0 | 88 | 1.4567 | 0.5909 | 0.3546 | 0.5348 | 0.5336 | 0.8879 | 0.9014 | 11.9357 | 18 | 6 | 17.1571 |
| 0.9413 | 12.0 | 96 | 1.4540 | 0.5881 | 0.3533 | 0.5351 | 0.5342 | 0.8879 | 0.9015 | 11.9571 | 18 | 6 | 17.25 |
| 0.9344 | 13.0 | 104 | 1.4489 | 0.5904 | 0.3602 | 0.5388 | 0.5374 | 0.8879 | 0.9013 | 11.9714 | 18 | 6 | 17.2643 |
| 0.929 | 14.0 | 112 | 1.4399 | 0.5866 | 0.355 | 0.5348 | 0.5338 | 0.8877 | 0.9006 | 11.8929 | 18 | 6 | 17.1857 |
| 0.9118 | 15.0 | 120 | 1.4353 | 0.5885 | 0.3569 | 0.537 | 0.5362 | 0.8883 | 0.9004 | 11.8 | 18 | 6 | 17.0857 |
| 0.9075 | 16.0 | 128 | 1.4326 | 0.5862 | 0.3531 | 0.5337 | 0.5329 | 0.8875 | 0.8998 | 11.8286 | 18 | 6 | 17.1143 |
| 0.9217 | 17.0 | 136 | 1.4296 | 0.5841 | 0.3547 | 0.534 | 0.5331 | 0.8882 | 0.9 | 11.7929 | 18 | 6 | 17.0571 |
| 0.8936 | 18.0 | 144 | 1.4270 | 0.5856 | 0.3558 | 0.5356 | 0.5347 | 0.8888 | 0.9003 | 11.75 | 18 | 6 | 17.0143 |
| 0.8848 | 19.0 | 152 | 1.4262 | 0.587 | 0.3564 | 0.5369 | 0.5357 | 0.8884 | 0.9005 | 11.8214 | 18 | 6 | 17.0857 |
| 0.8913 | 20.0 | 160 | 1.4263 | 0.587 | 0.3563 | 0.5367 | 0.5356 | 0.8882 | 0.9005 | 11.8286 | 18 | 6 | 17.0929 |
Framework versions
- Transformers 4.32.1
- Pytorch 2.0.1+cu118
- Datasets 2.14.4
- Tokenizers 0.13.3
