<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
DuplicatiDistillBertFullTraining
This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.4670
- Accuracy: 0.8904
- F1 Macro: 0.8349
- F1 Class 0: 0.9526
- F1 Class 1: 0.6667
- F1 Class 2: 0.8398
- F1 Class 3: 0.8278
- F1 Class 4: 0.8050
- F1 Class 5: 0.9111
- F1 Class 6: 0.8943
- F1 Class 7: 0.9504
- F1 Class 8: 0.6667
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Macro | F1 Class 0 | F1 Class 1 | F1 Class 2 | F1 Class 3 | F1 Class 4 | F1 Class 5 | F1 Class 6 | F1 Class 7 | F1 Class 8 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1.3064 | 0.25 | 250 | 0.7912 | 0.7411 | 0.5153 | 0.9353 | 0.0 | 0.5769 | 0.0 | 0.5222 | 0.8352 | 0.8477 | 0.9206 | 0.0 |
0.7377 | 0.5 | 500 | 0.6851 | 0.8024 | 0.6114 | 0.9458 | 0.0 | 0.6388 | 0.6040 | 0.6406 | 0.8646 | 0.8772 | 0.9313 | 0.0 |
0.5968 | 0.75 | 750 | 0.5917 | 0.8421 | 0.6460 | 0.9474 | 0.0 | 0.7722 | 0.7052 | 0.6909 | 0.8887 | 0.8812 | 0.9281 | 0.0 |
0.5028 | 1.01 | 1000 | 0.5893 | 0.8502 | 0.6523 | 0.9476 | 0.0 | 0.7700 | 0.7263 | 0.7564 | 0.8674 | 0.8537 | 0.9497 | 0.0 |
0.4657 | 1.26 | 1250 | 0.5319 | 0.8663 | 0.6671 | 0.9493 | 0.0 | 0.7830 | 0.7870 | 0.7650 | 0.8965 | 0.8777 | 0.9457 | 0.0 |
0.4047 | 1.51 | 1500 | 0.5214 | 0.8708 | 0.7452 | 0.9492 | 0.0 | 0.8141 | 0.7774 | 0.7784 | 0.8755 | 0.8978 | 0.9477 | 0.6667 |
0.4021 | 1.76 | 1750 | 0.5208 | 0.8773 | 0.7344 | 0.9476 | 0.0 | 0.7609 | 0.7879 | 0.8015 | 0.9156 | 0.8945 | 0.9563 | 0.5455 |
0.4 | 2.01 | 2000 | 0.4734 | 0.8879 | 0.8306 | 0.9527 | 0.6667 | 0.8274 | 0.8047 | 0.7965 | 0.9217 | 0.8856 | 0.9531 | 0.6667 |
0.2616 | 2.26 | 2250 | 0.5733 | 0.8763 | 0.7283 | 0.9577 | 0.0 | 0.7973 | 0.7926 | 0.8100 | 0.9012 | 0.8978 | 0.9278 | 0.4706 |
0.3004 | 2.52 | 2500 | 0.5050 | 0.8934 | 0.7959 | 0.9672 | 0.3333 | 0.8480 | 0.8235 | 0.8051 | 0.9149 | 0.8903 | 0.9556 | 0.625 |
0.3136 | 2.77 | 2750 | 0.4735 | 0.8894 | 0.8483 | 0.9511 | 0.9091 | 0.8444 | 0.7893 | 0.7992 | 0.9186 | 0.9 | 0.9514 | 0.5714 |
0.3091 | 3.02 | 3000 | 0.4670 | 0.8904 | 0.8349 | 0.9526 | 0.6667 | 0.8398 | 0.8278 | 0.8050 | 0.9111 | 0.8943 | 0.9504 | 0.6667 |
0.1983 | 3.27 | 3250 | 0.5770 | 0.8914 | 0.8328 | 0.9551 | 0.7500 | 0.8478 | 0.7956 | 0.8120 | 0.9156 | 0.8884 | 0.9598 | 0.5714 |
0.1782 | 3.52 | 3500 | 0.5193 | 0.8974 | 0.8245 | 0.9511 | 0.5714 | 0.8410 | 0.8353 | 0.8225 | 0.9196 | 0.9123 | 0.9521 | 0.6154 |
0.2419 | 3.77 | 3750 | 0.4857 | 0.8949 | 0.8129 | 0.9567 | 0.5 | 0.8495 | 0.7988 | 0.8177 | 0.9209 | 0.8980 | 0.9587 | 0.6154 |
0.2209 | 4.02 | 4000 | 0.5167 | 0.8994 | 0.7900 | 0.9501 | 0.3333 | 0.8509 | 0.8134 | 0.8345 | 0.9215 | 0.9112 | 0.9621 | 0.5333 |
0.1367 | 4.28 | 4250 | 0.6125 | 0.8919 | 0.8537 | 0.9582 | 0.8889 | 0.8411 | 0.8144 | 0.8190 | 0.9066 | 0.8820 | 0.9580 | 0.6154 |
0.1523 | 4.53 | 4500 | 0.5453 | 0.8944 | 0.8287 | 0.9565 | 0.7500 | 0.8404 | 0.8249 | 0.8155 | 0.9147 | 0.9002 | 0.9561 | 0.5 |
0.1666 | 4.78 | 4750 | 0.5185 | 0.9025 | 0.8497 | 0.9713 | 0.6667 | 0.8392 | 0.8394 | 0.8306 | 0.9226 | 0.9027 | 0.9601 | 0.7143 |
0.1388 | 5.03 | 5000 | 0.5815 | 0.8934 | 0.7865 | 0.9583 | 0.3333 | 0.8462 | 0.8288 | 0.8217 | 0.9126 | 0.8908 | 0.9604 | 0.5263 |
0.1039 | 5.28 | 5250 | 0.6477 | 0.8929 | 0.8184 | 0.9533 | 0.5 | 0.8431 | 0.8239 | 0.8103 | 0.9150 | 0.8913 | 0.9616 | 0.6667 |
0.0942 | 5.53 | 5500 | 0.6873 | 0.8864 | 0.8112 | 0.9603 | 0.6667 | 0.8424 | 0.8033 | 0.8031 | 0.9017 | 0.8914 | 0.9559 | 0.4762 |
0.1063 | 5.78 | 5750 | 0.6684 | 0.8944 | 0.8325 | 0.9675 | 0.5714 | 0.8557 | 0.8120 | 0.8204 | 0.9082 | 0.8884 | 0.9547 | 0.7143 |
0.0945 | 6.04 | 6000 | 0.6209 | 0.8939 | 0.8183 | 0.9654 | 0.5714 | 0.8537 | 0.8184 | 0.8112 | 0.9175 | 0.8982 | 0.9405 | 0.5882 |
0.0771 | 6.29 | 6250 | 0.6268 | 0.8994 | 0.8563 | 0.9638 | 0.7500 | 0.8398 | 0.8363 | 0.8373 | 0.9123 | 0.8924 | 0.9605 | 0.7143 |
0.0845 | 6.54 | 6500 | 0.6382 | 0.8939 | 0.8417 | 0.9692 | 0.7500 | 0.8429 | 0.8179 | 0.8151 | 0.9123 | 0.8884 | 0.9548 | 0.625 |
0.0673 | 6.79 | 6750 | 0.6561 | 0.9010 | 0.8315 | 0.9693 | 0.5714 | 0.8404 | 0.8214 | 0.8342 | 0.9252 | 0.8928 | 0.9616 | 0.6667 |
0.0641 | 7.04 | 7000 | 0.7066 | 0.8879 | 0.8407 | 0.9617 | 0.7500 | 0.8467 | 0.7923 | 0.8107 | 0.9077 | 0.8795 | 0.9512 | 0.6667 |
0.039 | 7.29 | 7250 | 0.6932 | 0.8949 | 0.8459 | 0.9659 | 0.7500 | 0.8510 | 0.8079 | 0.8178 | 0.9185 | 0.8767 | 0.9590 | 0.6667 |
0.0372 | 7.55 | 7500 | 0.6786 | 0.8984 | 0.8705 | 0.9658 | 0.8889 | 0.8626 | 0.8232 | 0.8194 | 0.9134 | 0.8859 | 0.9607 | 0.7143 |
0.0504 | 7.8 | 7750 | 0.6914 | 0.8949 | 0.8598 | 0.9641 | 0.9091 | 0.8478 | 0.8202 | 0.8104 | 0.9177 | 0.8874 | 0.9561 | 0.625 |
0.0409 | 8.05 | 8000 | 0.7027 | 0.8984 | 0.8501 | 0.9658 | 0.7500 | 0.8475 | 0.8387 | 0.8195 | 0.9142 | 0.8879 | 0.9607 | 0.6667 |
0.0196 | 8.3 | 8250 | 0.7222 | 0.8969 | 0.8530 | 0.9659 | 0.7500 | 0.8492 | 0.8202 | 0.8123 | 0.9184 | 0.8849 | 0.9621 | 0.7143 |
0.0323 | 8.55 | 8500 | 0.6858 | 0.8999 | 0.8551 | 0.9697 | 0.8889 | 0.8606 | 0.8235 | 0.8218 | 0.9181 | 0.9015 | 0.9561 | 0.5556 |
0.0274 | 8.8 | 8750 | 0.6813 | 0.9010 | 0.8557 | 0.9660 | 0.8889 | 0.8517 | 0.8300 | 0.8270 | 0.9186 | 0.9015 | 0.9618 | 0.5556 |
0.0212 | 9.05 | 9000 | 0.7197 | 0.8979 | 0.8608 | 0.9677 | 0.8889 | 0.8456 | 0.8272 | 0.8281 | 0.9111 | 0.8899 | 0.9633 | 0.625 |
0.0065 | 9.31 | 9250 | 0.7363 | 0.8979 | 0.8601 | 0.9696 | 0.8889 | 0.8463 | 0.8199 | 0.8220 | 0.9152 | 0.8924 | 0.9618 | 0.625 |
0.0115 | 9.56 | 9500 | 0.7331 | 0.8974 | 0.8647 | 0.9677 | 0.8889 | 0.8504 | 0.8249 | 0.8204 | 0.9105 | 0.8909 | 0.9619 | 0.6667 |
0.0059 | 9.81 | 9750 | 0.7349 | 0.8989 | 0.8660 | 0.9695 | 0.8889 | 0.8462 | 0.8319 | 0.8226 | 0.9121 | 0.8953 | 0.9606 | 0.6667 |
Framework versions
- Transformers 4.32.1
- Pytorch 2.0.1+cu117
- Datasets 2.14.4
- Tokenizers 0.13.3