<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
8_koelectra_train_korquad-1_2_aihub
This model is a fine-tuned version of monologg/koelectra-base-v3-discriminator on the None dataset. It achieves the following results on the evaluation set:
- Exact Match: 78.9613
- F1: 84.5790
- Loss: 0.8506
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 10
- mixed_precision_training: Native AMP
- label_smoothing_factor: 0.1
Training results
| Training Loss | Epoch | Step | Exact Match | F1 | Validation Loss |
|---|---|---|---|---|---|
| 1.5282 | 0.33 | 4000 | 64.4807 | 72.3547 | 1.4717 |
| 1.0638 | 0.67 | 8000 | 72.1118 | 79.0236 | 1.0938 |
| 1.0703 | 1.0 | 12000 | 74.0859 | 80.5242 | 1.0459 |
| 0.9242 | 1.34 | 16000 | 75.1325 | 81.4470 | 0.9775 |
| 0.9312 | 1.67 | 20000 | 75.6492 | 81.7357 | 0.9707 |
| 0.9483 | 2.01 | 24000 | 76.2189 | 82.3461 | 0.9248 |
| 0.8454 | 2.34 | 28000 | 76.8813 | 82.9913 | 0.9268 |
| 0.8541 | 2.67 | 32000 | 77.1330 | 83.1591 | 0.9004 |
| 0.8647 | 3.01 | 36000 | 77.1860 | 83.1519 | 0.8911 |
| 0.8952 | 3.34 | 40000 | 77.1993 | 83.1777 | 0.8765 |
| 0.7345 | 3.68 | 44000 | 77.3450 | 83.4184 | 0.9365 |
| 0.708 | 4.01 | 48000 | 77.8617 | 83.7737 | 0.8599 |
| 0.7217 | 4.34 | 52000 | 77.8352 | 83.6681 | 0.8770 |
| 0.817 | 4.68 | 56000 | 77.9809 | 83.8054 | 0.8730 |
| 0.7655 | 5.01 | 60000 | 78.0207 | 83.8704 | 0.8623 |
| 0.7276 | 5.35 | 64000 | 78.2989 | 84.0245 | 0.8535 |
| 0.6739 | 5.68 | 68000 | 78.2724 | 84.0880 | 0.8726 |
| 0.652 | 6.02 | 72000 | 78.5639 | 84.2059 | 0.8657 |
| 0.6615 | 6.35 | 76000 | 78.3254 | 84.1279 | 0.8623 |
| 0.6624 | 6.68 | 80000 | 78.7493 | 84.4215 | 0.8525 |
| 0.707 | 7.02 | 84000 | 78.5374 | 84.2300 | 0.8486 |
| 0.8086 | 7.35 | 88000 | 78.3519 | 84.1909 | 0.8442 |
| 0.6347 | 7.69 | 92000 | 78.6963 | 84.4347 | 0.8760 |
| 0.702 | 8.02 | 96000 | 78.9083 | 84.6330 | 0.8418 |
| 0.6618 | 8.36 | 100000 | 78.7493 | 84.5021 | 0.8672 |
| 0.6294 | 8.69 | 104000 | 78.5374 | 84.3771 | 0.8770 |
| 0.5797 | 9.02 | 108000 | 78.5904 | 84.3051 | 0.8623 |
| 0.6073 | 9.36 | 112000 | 78.9216 | 84.6703 | 0.8638 |
| 0.6717 | 9.69 | 116000 | 78.9613 | 84.5790 | 0.8506 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.7.1
- Datasets 2.7.0
- Tokenizers 0.13.2
