<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
koelectra-small-v2-discriminator-finetuned-korspeech-ser
This model is a fine-tuned version of monologg/koelectra-small-v2-discriminator on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.6832
- Macro F1: 0.7998
- Accuracy: 0.7997
- Weighted f1: 0.7998
- Micro f1: 0.7997
- Weighted recall: 0.7997
- Micro recall: 0.7997
- Macro recall: 0.7997
- Weighted precision: 0.8000
- Micro precision: 0.7997
- Macro precision: 0.8000
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- lr_scheduler_warmup_steps: 100
- num_epochs: 10
Training results
| Training Loss | Epoch | Step | Validation Loss | Macro F1 | Accuracy | Weighted f1 | Micro f1 | Weighted recall | Micro recall | Macro recall | Weighted precision | Micro precision | Macro precision |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1.3255 | 0.57 | 100 | 1.0748 | 0.5051 | 0.5589 | 0.5051 | 0.5589 | 0.5589 | 0.5589 | 0.5589 | 0.5498 | 0.5589 | 0.5498 |
| 0.9496 | 1.13 | 200 | 0.7828 | 0.7228 | 0.7216 | 0.7228 | 0.7216 | 0.7216 | 0.7216 | 0.7216 | 0.7285 | 0.7216 | 0.7285 |
| 0.7587 | 1.7 | 300 | 0.6639 | 0.7496 | 0.75 | 0.7496 | 0.75 | 0.75 | 0.75 | 0.75 | 0.7516 | 0.75 | 0.7516 |
| 0.6209 | 2.27 | 400 | 0.6279 | 0.7720 | 0.7713 | 0.7720 | 0.7713 | 0.7713 | 0.7713 | 0.7713 | 0.7766 | 0.7713 | 0.7766 |
| 0.5722 | 2.84 | 500 | 0.6025 | 0.7840 | 0.7841 | 0.7840 | 0.7841 | 0.7841 | 0.7841 | 0.7841 | 0.7840 | 0.7841 | 0.7840 |
| 0.5125 | 3.4 | 600 | 0.5931 | 0.7886 | 0.7891 | 0.7886 | 0.7891 | 0.7891 | 0.7891 | 0.7891 | 0.7894 | 0.7891 | 0.7894 |
| 0.4742 | 3.97 | 700 | 0.5733 | 0.7926 | 0.7912 | 0.7926 | 0.7912 | 0.7912 | 0.7912 | 0.7912 | 0.7953 | 0.7912 | 0.7953 |
| 0.4124 | 4.54 | 800 | 0.6031 | 0.7928 | 0.7933 | 0.7928 | 0.7933 | 0.7933 | 0.7933 | 0.7933 | 0.7955 | 0.7933 | 0.7955 |
| 0.3933 | 5.11 | 900 | 0.6151 | 0.7927 | 0.7926 | 0.7927 | 0.7926 | 0.7926 | 0.7926 | 0.7926 | 0.7933 | 0.7926 | 0.7933 |
| 0.3429 | 5.67 | 1000 | 0.6393 | 0.7913 | 0.7905 | 0.7913 | 0.7905 | 0.7905 | 0.7905 | 0.7905 | 0.7980 | 0.7905 | 0.7980 |
| 0.3362 | 6.24 | 1100 | 0.6386 | 0.7920 | 0.7940 | 0.7920 | 0.7940 | 0.7940 | 0.7940 | 0.7940 | 0.7932 | 0.7940 | 0.7932 |
| 0.3125 | 6.81 | 1200 | 0.6291 | 0.8044 | 0.8047 | 0.8044 | 0.8047 | 0.8047 | 0.8047 | 0.8047 | 0.8044 | 0.8047 | 0.8044 |
| 0.2758 | 7.38 | 1300 | 0.6402 | 0.7981 | 0.7976 | 0.7981 | 0.7976 | 0.7976 | 0.7976 | 0.7976 | 0.7995 | 0.7976 | 0.7995 |
| 0.2664 | 7.94 | 1400 | 0.6570 | 0.8050 | 0.8047 | 0.8050 | 0.8047 | 0.8047 | 0.8047 | 0.8047 | 0.8058 | 0.8047 | 0.8058 |
| 0.2451 | 8.51 | 1500 | 0.6735 | 0.7940 | 0.7933 | 0.7940 | 0.7933 | 0.7933 | 0.7933 | 0.7933 | 0.7960 | 0.7933 | 0.7960 |
| 0.236 | 9.08 | 1600 | 0.6844 | 0.7998 | 0.8004 | 0.7998 | 0.8004 | 0.8004 | 0.8004 | 0.8004 | 0.7997 | 0.8004 | 0.7997 |
| 0.2236 | 9.65 | 1700 | 0.6832 | 0.7998 | 0.7997 | 0.7998 | 0.7997 | 0.7997 | 0.7997 | 0.7997 | 0.8000 | 0.7997 | 0.8000 |
Framework versions
- Transformers 4.30.2
- Pytorch 2.0.0
- Datasets 2.1.0
- Tokenizers 0.13.3
