<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
t5-fine-tuned-with-yake-keywords
This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.7156
- Rouge1: 25.7203
- Rouge2: 11.4526
- Rougel: 20.9169
- Rougelsum: 24.2292
- Gen Len: 19.0
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 25
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| 1.408 | 1.0 | 604 | 1.3840 | 25.8101 | 11.1365 | 20.9036 | 24.1453 | 18.992 |
| 1.2639 | 2.0 | 1208 | 1.3853 | 25.3277 | 11.0463 | 20.7012 | 23.7892 | 18.994 |
| 1.2049 | 3.0 | 1812 | 1.3945 | 25.7271 | 11.437 | 20.9819 | 24.2143 | 18.99 |
| 1.1008 | 4.0 | 2416 | 1.4060 | 25.4068 | 10.9722 | 20.3943 | 23.7708 | 18.994 |
| 1.0033 | 5.0 | 3020 | 1.4295 | 25.0656 | 10.7868 | 20.1689 | 23.4475 | 18.992 |
| 0.9373 | 6.0 | 3624 | 1.4425 | 25.5643 | 11.1203 | 20.5504 | 23.8398 | 19.0 |
| 0.9054 | 7.0 | 4228 | 1.4600 | 25.6599 | 11.3601 | 20.7753 | 24.0562 | 19.0 |
| 0.8699 | 8.0 | 4832 | 1.4881 | 25.8057 | 11.6887 | 20.9235 | 24.2292 | 19.0 |
| 0.8253 | 9.0 | 5436 | 1.5049 | 25.5996 | 11.2568 | 20.6128 | 23.9682 | 19.0 |
| 0.7706 | 10.0 | 6040 | 1.5304 | 25.5366 | 11.2432 | 20.5506 | 23.8558 | 19.0 |
| 0.7406 | 11.0 | 6644 | 1.5443 | 25.5608 | 11.3835 | 20.7609 | 23.9875 | 19.0 |
| 0.7122 | 12.0 | 7248 | 1.5608 | 25.4136 | 11.2439 | 20.6605 | 23.9639 | 19.0 |
| 0.6983 | 13.0 | 7852 | 1.5777 | 26.1446 | 11.7678 | 21.1421 | 24.587 | 19.0 |
| 0.6506 | 14.0 | 8456 | 1.5987 | 26.0036 | 11.8495 | 21.1601 | 24.5154 | 19.0 |
| 0.627 | 15.0 | 9060 | 1.6229 | 25.7103 | 11.5427 | 20.9998 | 24.2446 | 19.0 |
| 0.625 | 16.0 | 9664 | 1.6448 | 25.4558 | 11.4994 | 20.82 | 23.9969 | 19.0 |
| 0.5918 | 17.0 | 10268 | 1.6613 | 25.8231 | 11.6828 | 20.9607 | 24.2662 | 19.0 |
| 0.577 | 18.0 | 10872 | 1.6786 | 25.9623 | 11.6631 | 21.0312 | 24.4392 | 19.0 |
| 0.5917 | 19.0 | 11476 | 1.6810 | 25.9725 | 11.5125 | 20.9832 | 24.4222 | 19.0 |
| 0.5543 | 20.0 | 12080 | 1.6896 | 25.8324 | 11.5314 | 20.9358 | 24.2487 | 19.0 |
| 0.5551 | 21.0 | 12684 | 1.7021 | 25.753 | 11.5405 | 21.022 | 24.2551 | 19.0 |
| 0.5334 | 22.0 | 13288 | 1.7114 | 25.7559 | 11.4908 | 20.9003 | 24.2192 | 19.0 |
| 0.5301 | 23.0 | 13892 | 1.7137 | 25.7977 | 11.4324 | 20.8802 | 24.241 | 19.0 |
| 0.5333 | 24.0 | 14496 | 1.7150 | 25.7444 | 11.4107 | 20.8701 | 24.205 | 19.0 |
| 0.5216 | 25.0 | 15100 | 1.7156 | 25.7203 | 11.4526 | 20.9169 | 24.2292 | 19.0 |
Framework versions
- Transformers 4.33.3
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.13.3
