<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
Vigec-V3
This model is a fine-tuned version of VietAI/vit5-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.2522
- Bleu: 84.4788
- Gen Len: 9.847
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- training_steps: 10000
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
|---|---|---|---|---|---|
| 0.8764 | 0.0 | 500 | 0.6120 | 68.0114 | 8.4626 |
| 0.6538 | 0.0 | 1000 | 0.4780 | 76.7403 | 10.015 |
| 0.6234 | 0.01 | 1500 | 0.4207 | 78.1726 | 9.8394 |
| 0.4513 | 0.01 | 2000 | 0.3845 | 79.1939 | 9.8914 |
| 0.4153 | 0.01 | 2500 | 0.3580 | 80.171 | 9.7298 |
| 0.5129 | 0.01 | 3000 | 0.3381 | 80.8668 | 9.8636 |
| 0.5073 | 0.01 | 3500 | 0.3246 | 81.5543 | 9.81 |
| 0.4623 | 0.01 | 4000 | 0.3106 | 82.1255 | 9.8684 |
| 0.4444 | 0.02 | 4500 | 0.2973 | 82.5565 | 9.848 |
| 0.4322 | 0.02 | 5000 | 0.2892 | 82.9623 | 9.872 |
| 0.5029 | 0.02 | 5500 | 0.2803 | 83.3084 | 9.8648 |
| 0.3686 | 0.02 | 6000 | 0.2765 | 83.4828 | 9.8602 |
| 0.4123 | 0.02 | 6500 | 0.2693 | 83.7491 | 9.8432 |
| 0.3593 | 0.03 | 7000 | 0.2674 | 83.8149 | 9.811 |
| 0.3684 | 0.03 | 7500 | 0.2630 | 84.1745 | 9.8668 |
| 0.3683 | 0.03 | 8000 | 0.2590 | 84.2294 | 9.8412 |
| 0.3581 | 0.03 | 8500 | 0.2568 | 84.3428 | 9.8582 |
| 0.3769 | 0.03 | 9000 | 0.2527 | 84.4367 | 9.8598 |
| 0.4479 | 0.03 | 9500 | 0.2522 | 84.4749 | 9.847 |
| 0.2856 | 0.04 | 10000 | 0.2522 | 84.4788 | 9.847 |
Framework versions
- Transformers 4.26.0
- Pytorch 1.13.1+cu116
- Datasets 2.9.0
- Tokenizers 0.13.2
