<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
fugumt-en-ja-finetuned-en-to-ja-19962
This model is a fine-tuned version of staka/fugumt-en-ja on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.0284
- Bleu: 94.9277
- Gen Len: 8.2017
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 25
Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
|---|---|---|---|---|---|
| 1.3412 | 1.0 | 1248 | 0.9566 | 37.9316 | 8.0629 |
| 1.0091 | 2.0 | 2496 | 0.7058 | 44.472 | 8.3194 |
| 0.7897 | 3.0 | 3744 | 0.5457 | 50.2279 | 8.3457 |
| 0.6493 | 4.0 | 4992 | 0.4351 | 60.5711 | 8.3049 |
| 0.5295 | 5.0 | 6240 | 0.3471 | 65.7518 | 8.1633 |
| 0.4446 | 6.0 | 7488 | 0.2841 | 64.7884 | 8.4418 |
| 0.3819 | 7.0 | 8736 | 0.2302 | 72.3781 | 8.3635 |
| 0.3182 | 8.0 | 9984 | 0.1867 | 76.0959 | 8.3377 |
| 0.2832 | 9.0 | 11232 | 0.1536 | 77.415 | 8.2123 |
| 0.2415 | 10.0 | 12480 | 0.1286 | 82.0799 | 8.1812 |
| 0.2178 | 11.0 | 13728 | 0.1064 | 84.947 | 8.1949 |
| 0.1869 | 12.0 | 14976 | 0.0890 | 87.698 | 8.1957 |
| 0.1684 | 13.0 | 16224 | 0.0752 | 90.0452 | 8.1591 |
| 0.146 | 14.0 | 17472 | 0.0667 | 90.3098 | 8.211 |
| 0.1353 | 15.0 | 18720 | 0.0576 | 91.5242 | 8.166 |
| 0.1192 | 16.0 | 19968 | 0.0500 | 92.8459 | 8.2385 |
| 0.1132 | 17.0 | 21216 | 0.0445 | 93.437 | 8.2262 |
| 0.101 | 18.0 | 22464 | 0.0402 | 94.0457 | 8.1928 |
| 0.0987 | 19.0 | 23712 | 0.0368 | 94.4763 | 8.1896 |
| 0.0848 | 20.0 | 24960 | 0.0343 | 94.7051 | 8.1926 |
| 0.0814 | 21.0 | 26208 | 0.0324 | 94.411 | 8.2673 |
| 0.0802 | 22.0 | 27456 | 0.0310 | 94.5696 | 8.2017 |
| 0.0733 | 23.0 | 28704 | 0.0296 | 94.8238 | 8.2035 |
| 0.0704 | 24.0 | 29952 | 0.0287 | 94.8744 | 8.2018 |
| 0.0708 | 25.0 | 31200 | 0.0284 | 94.9277 | 8.2017 |
Framework versions
- Transformers 4.34.0
- Pytorch 2.1.0+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1
