<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
t5-text-simplification_1e4_adafactor_biendata
This model is a fine-tuned version of t5-base on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.7562
- Rouge1: 10.4603
- Rouge2: 2.642
- Rougel: 9.6362
- Rougelsum: 9.6589
- Gen Len: 13.2838
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 464 | 0.5489 | 29.7693 | 11.1997 | 25.6091 | 25.5979 | 14.7281 |
| 0.9314 | 2.0 | 928 | 0.5392 | 29.9099 | 10.9645 | 25.334 | 25.3259 | 14.7188 |
| 0.5594 | 3.0 | 1392 | 0.5342 | 30.3194 | 11.4204 | 25.8248 | 25.8255 | 14.7666 |
| 0.5333 | 4.0 | 1856 | 0.5376 | 30.8368 | 11.6152 | 26.3172 | 26.3583 | 14.1578 |
| 0.5192 | 5.0 | 2320 | 0.8890 | 7.5517 | 1.4313 | 7.0971 | 7.1064 | 9.9191 |
| 0.8897 | 6.0 | 2784 | 0.8252 | 6.9283 | 1.3484 | 6.5916 | 6.5877 | 10.9894 |
| 0.9385 | 7.0 | 3248 | 0.7971 | 8.2401 | 1.9957 | 7.7693 | 7.7675 | 10.7732 |
| 0.9089 | 8.0 | 3712 | 0.7725 | 9.7559 | 2.2249 | 9.0272 | 9.0098 | 10.7175 |
| 0.8824 | 9.0 | 4176 | 0.7552 | 12.006 | 2.8041 | 11.0115 | 10.992 | 10.7838 |
| 0.8658 | 10.0 | 4640 | 0.7490 | 13.311 | 3.4159 | 12.1933 | 12.1551 | 10.6499 |
| 0.864 | 11.0 | 5104 | 0.7448 | 13.9983 | 3.6176 | 12.7712 | 12.7347 | 10.752 |
| 0.868 | 12.0 | 5568 | 0.7495 | 12.318 | 3.2975 | 11.3451 | 11.3218 | 12.0252 |
| 0.8844 | 13.0 | 6032 | 0.7552 | 10.6154 | 2.7347 | 9.8228 | 9.8116 | 13.191 |
| 0.8844 | 14.0 | 6496 | 0.7562 | 10.4603 | 2.642 | 9.6362 | 9.6589 | 13.2838 |
| 0.8971 | 15.0 | 6960 | 0.7562 | 10.4603 | 2.642 | 9.6362 | 9.6589 | 13.2838 |
| 0.8981 | 16.0 | 7424 | 0.7562 | 10.4603 | 2.642 | 9.6362 | 9.6589 | 13.2838 |
| 0.8956 | 17.0 | 7888 | 0.7562 | 10.4603 | 2.642 | 9.6362 | 9.6589 | 13.2838 |
| 0.8984 | 18.0 | 8352 | 0.7562 | 10.4603 | 2.642 | 9.6362 | 9.6589 | 13.2838 |
| 0.8959 | 19.0 | 8816 | 0.7562 | 10.4603 | 2.642 | 9.6362 | 9.6589 | 13.2838 |
| 0.8977 | 20.0 | 9280 | 0.7562 | 10.4603 | 2.642 | 9.6362 | 9.6589 | 13.2838 |
Framework versions
- Transformers 4.30.2
- Pytorch 2.0.1+cu118
- Datasets 2.13.1
- Tokenizers 0.13.3
