<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
opus-mt-en-id-ccmatrix-lr-4
This model was trained from scratch on the ccmatrix dataset. It achieves the following results on the evaluation set:
- Loss: 0.9115
- Bleu: 65.4544
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 4000
- num_epochs: 25
Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu |
|---|---|---|---|---|
| 0.7279 | 1.0 | 28125 | 0.7185 | 61.1164 |
| 0.6185 | 2.0 | 56250 | 0.6849 | 62.0536 |
| 0.5598 | 3.0 | 84375 | 0.6759 | 62.3915 |
| 0.5163 | 4.0 | 112500 | 0.6646 | 62.9303 |
| 0.4795 | 5.0 | 140625 | 0.6665 | 63.3461 |
| 0.4471 | 6.0 | 168750 | 0.6692 | 63.5319 |
| 0.4173 | 7.0 | 196875 | 0.6690 | 63.7436 |
| 0.3897 | 8.0 | 225000 | 0.6739 | 63.8343 |
| 0.3633 | 9.0 | 253125 | 0.6832 | 63.867 |
| 0.3382 | 10.0 | 281250 | 0.6928 | 64.0481 |
| 0.314 | 11.0 | 309375 | 0.7015 | 64.0177 |
| 0.2909 | 12.0 | 337500 | 0.7151 | 64.3563 |
| 0.2687 | 13.0 | 365625 | 0.7265 | 64.2445 |
| 0.2474 | 14.0 | 393750 | 0.7384 | 64.5093 |
| 0.227 | 15.0 | 421875 | 0.7560 | 64.3729 |
| 0.2072 | 16.0 | 450000 | 0.7712 | 64.6396 |
| 0.1888 | 17.0 | 478125 | 0.7876 | 64.805 |
| 0.1713 | 18.0 | 506250 | 0.8052 | 64.7883 |
| 0.1546 | 19.0 | 534375 | 0.8258 | 64.9535 |
| 0.1394 | 20.0 | 562500 | 0.8421 | 64.9885 |
| 0.1251 | 21.0 | 590625 | 0.8593 | 65.1229 |
| 0.112 | 22.0 | 618750 | 0.8757 | 65.2565 |
| 0.1006 | 23.0 | 646875 | 0.8923 | 65.288 |
| 0.0907 | 24.0 | 675000 | 0.9033 | 65.3973 |
| 0.0828 | 25.0 | 703125 | 0.9115 | 65.4544 |
Framework versions
- Transformers 4.26.1
- Pytorch 2.0.0
- Datasets 2.10.1
- Tokenizers 0.11.0
