<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
gpt_16_5_5.6e-5
This model is a fine-tuned version of skt/kogpt2-base-v2 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 4.0127
- Rouge1: 0.6626
- Rouge2: 0.1168
- Rougel: 0.6456
- Bleu1: 0.0
- Bleu2: 0.0
- Bleu3: 0.0
- Bleu4: 0.0
- Gen Len: 1.0
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5.6e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 5.0
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Bleu1 | Bleu2 | Bleu3 | Bleu4 | Gen Len |
---|---|---|---|---|---|---|---|---|---|---|---|
3.1501 | 0.38 | 1000 | 4.1034 | 0.6908 | 0.1016 | 0.6736 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
3.1688 | 0.76 | 2000 | 4.0936 | 0.6447 | 0.1117 | 0.6372 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.8769 | 1.13 | 3000 | 4.0668 | 0.7186 | 0.1365 | 0.7012 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.8633 | 1.51 | 4000 | 4.0363 | 0.625 | 0.1187 | 0.6116 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.8928 | 1.89 | 5000 | 4.0112 | 0.6599 | 0.1237 | 0.6476 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.6186 | 2.27 | 6000 | 4.0204 | 0.6953 | 0.1259 | 0.6761 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.6653 | 2.64 | 7000 | 4.0044 | 0.6795 | 0.1163 | 0.6627 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.5418 | 3.02 | 8000 | 4.0183 | 0.602 | 0.1154 | 0.5883 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.4621 | 3.4 | 9000 | 4.0149 | 0.5889 | 0.101 | 0.578 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.4651 | 3.78 | 10000 | 3.9979 | 0.604 | 0.1098 | 0.5943 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.3334 | 4.15 | 11000 | 4.0226 | 0.6352 | 0.0993 | 0.6211 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.2859 | 4.53 | 12000 | 4.0152 | 0.6782 | 0.1125 | 0.6593 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
2.314 | 4.91 | 13000 | 4.0127 | 0.6626 | 0.1168 | 0.6456 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.9.0+cu102
- Datasets 2.8.0
- Tokenizers 0.13.2