<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
mt5-small-finetuned-new3
This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 2.3994
- Rouge1: 20.61
- Rouge2: 6.06
- Rougel: 20.22
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 9
- eval_batch_size: 9
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 40
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel |
|---|---|---|---|---|---|---|
| 4.7204 | 1.45 | 500 | 2.6053 | 16.92 | 4.91 | 16.76 |
| 3.1289 | 2.9 | 1000 | 2.4878 | 18.01 | 5.24 | 17.83 |
| 2.8862 | 4.35 | 1500 | 2.4109 | 17.48 | 5.07 | 17.11 |
| 2.7669 | 5.8 | 2000 | 2.4006 | 18.57 | 5.26 | 18.22 |
| 2.6433 | 7.25 | 2500 | 2.4017 | 18.77 | 5.68 | 18.56 |
| 2.5514 | 8.7 | 3000 | 2.3917 | 19.38 | 5.9 | 19.11 |
| 2.4947 | 10.14 | 3500 | 2.3994 | 20.61 | 6.06 | 20.22 |
| 2.3995 | 11.59 | 4000 | 2.3608 | 20.13 | 6.5 | 19.78 |
| 2.3798 | 13.04 | 4500 | 2.3251 | 20.03 | 6.24 | 19.72 |
| 2.3029 | 14.49 | 5000 | 2.3387 | 19.69 | 6.13 | 19.44 |
| 2.2563 | 15.94 | 5500 | 2.3372 | 20.17 | 6.35 | 19.77 |
| 2.2109 | 17.39 | 6000 | 2.3410 | 20.58 | 6.36 | 20.1 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.1
- Tokenizers 0.13.2
