<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
fine_tuned_bert_dreadit
This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 2.6081
- Accuracy: 0.7528
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.0037 | 1.0 | 178 | 1.8515 | 0.7163 |
| 0.0017 | 2.0 | 356 | 1.7404 | 0.7163 |
| 0.001 | 3.0 | 534 | 1.2895 | 0.7921 |
| 0.0012 | 4.0 | 712 | 1.3320 | 0.7669 |
| 0.0005 | 5.0 | 890 | 1.3646 | 0.7949 |
| 0.0002 | 6.0 | 1068 | 1.5997 | 0.7809 |
| 0.0001 | 7.0 | 1246 | 1.5772 | 0.7753 |
| 0.0003 | 8.0 | 1424 | 1.7599 | 0.7556 |
| 0.0001 | 9.0 | 1602 | 1.7494 | 0.7640 |
| 0.0001 | 10.0 | 1780 | 1.9942 | 0.7556 |
| 0.0001 | 11.0 | 1958 | 1.9370 | 0.75 |
| 0.0 | 12.0 | 2136 | 1.9671 | 0.7781 |
| 0.0001 | 13.0 | 2314 | 2.1223 | 0.7640 |
| 0.0 | 14.0 | 2492 | 2.1653 | 0.7472 |
| 0.0001 | 15.0 | 2670 | 1.9924 | 0.75 |
| 0.0 | 16.0 | 2848 | 2.1778 | 0.7528 |
| 0.0 | 17.0 | 3026 | 2.3010 | 0.7612 |
| 0.0 | 18.0 | 3204 | 2.2210 | 0.7669 |
| 0.0 | 19.0 | 3382 | 2.3333 | 0.7556 |
| 0.0 | 20.0 | 3560 | 1.8684 | 0.7697 |
| 0.0976 | 21.0 | 3738 | 1.9417 | 0.7584 |
| 0.0 | 22.0 | 3916 | 2.1385 | 0.7472 |
| 0.0 | 23.0 | 4094 | 1.9774 | 0.7669 |
| 0.0 | 24.0 | 4272 | 2.0778 | 0.75 |
| 0.0001 | 25.0 | 4450 | 2.4343 | 0.7331 |
| 0.0 | 26.0 | 4628 | 2.1331 | 0.7528 |
| 0.0 | 27.0 | 4806 | 2.2511 | 0.7640 |
| 0.0 | 28.0 | 4984 | 2.2422 | 0.7584 |
| 0.0 | 29.0 | 5162 | 2.1228 | 0.7669 |
| 0.0006 | 30.0 | 5340 | 2.0973 | 0.7725 |
| 0.0 | 31.0 | 5518 | 1.9392 | 0.7809 |
| 0.0 | 32.0 | 5696 | 2.2996 | 0.7107 |
| 0.4186 | 33.0 | 5874 | 2.2191 | 0.7584 |
| 0.0 | 34.0 | 6052 | 2.2233 | 0.75 |
| 0.0 | 35.0 | 6230 | 2.2263 | 0.7584 |
| 0.0 | 36.0 | 6408 | 2.2205 | 0.7584 |
| 0.0 | 37.0 | 6586 | 2.4488 | 0.7444 |
| 0.0 | 38.0 | 6764 | 2.5616 | 0.7360 |
| 0.0 | 39.0 | 6942 | 2.5941 | 0.7416 |
| 0.0 | 40.0 | 7120 | 2.5129 | 0.7528 |
| 0.0 | 41.0 | 7298 | 2.4978 | 0.7360 |
| 0.0 | 42.0 | 7476 | 2.3089 | 0.7528 |
| 0.0 | 43.0 | 7654 | 2.5056 | 0.7472 |
| 0.0 | 44.0 | 7832 | 2.5786 | 0.7416 |
| 0.0 | 45.0 | 8010 | 2.2956 | 0.7640 |
| 0.0 | 46.0 | 8188 | 2.5265 | 0.7472 |
| 0.0 | 47.0 | 8366 | 2.4396 | 0.7584 |
| 0.0 | 48.0 | 8544 | 2.5547 | 0.7472 |
| 0.0 | 49.0 | 8722 | 2.5556 | 0.7528 |
| 0.0 | 50.0 | 8900 | 2.5732 | 0.7528 |
| 0.0 | 51.0 | 9078 | 2.5062 | 0.7556 |
| 0.0 | 52.0 | 9256 | 2.5504 | 0.7528 |
| 0.0 | 53.0 | 9434 | 2.5602 | 0.7528 |
| 0.0 | 54.0 | 9612 | 2.5627 | 0.7472 |
| 0.0 | 55.0 | 9790 | 2.6575 | 0.75 |
| 0.0 | 56.0 | 9968 | 2.6239 | 0.7528 |
| 0.0 | 57.0 | 10146 | 2.4757 | 0.7697 |
| 0.0 | 58.0 | 10324 | 2.4862 | 0.7612 |
| 0.0 | 59.0 | 10502 | 3.2968 | 0.6938 |
| 0.0 | 60.0 | 10680 | 2.5265 | 0.7472 |
| 0.0 | 61.0 | 10858 | 2.1426 | 0.7978 |
| 0.0 | 62.0 | 11036 | 2.4674 | 0.7640 |
| 0.0 | 63.0 | 11214 | 2.3496 | 0.7640 |
| 0.0 | 64.0 | 11392 | 2.4010 | 0.7556 |
| 0.0 | 65.0 | 11570 | 2.4081 | 0.7725 |
| 0.0 | 66.0 | 11748 | 2.4022 | 0.7753 |
| 0.0 | 67.0 | 11926 | 2.2982 | 0.7753 |
| 0.0 | 68.0 | 12104 | 2.4628 | 0.7612 |
| 0.0 | 69.0 | 12282 | 2.5764 | 0.7640 |
| 0.0 | 70.0 | 12460 | 2.4056 | 0.7781 |
| 0.0 | 71.0 | 12638 | 2.3265 | 0.7865 |
| 0.0 | 72.0 | 12816 | 2.5182 | 0.7640 |
| 0.0 | 73.0 | 12994 | 2.3872 | 0.7556 |
| 0.0 | 74.0 | 13172 | 2.7281 | 0.7388 |
| 0.0 | 75.0 | 13350 | 2.4907 | 0.7612 |
| 0.0 | 76.0 | 13528 | 2.5323 | 0.7584 |
| 0.0 | 77.0 | 13706 | 2.2055 | 0.7837 |
| 0.0 | 78.0 | 13884 | 2.2227 | 0.7865 |
| 0.0 | 79.0 | 14062 | 2.2794 | 0.7753 |
| 0.0 | 80.0 | 14240 | 2.2886 | 0.7753 |
| 0.0 | 81.0 | 14418 | 2.8320 | 0.7444 |
| 0.0 | 82.0 | 14596 | 2.8252 | 0.7472 |
| 0.0 | 83.0 | 14774 | 2.2986 | 0.7837 |
| 0.0 | 84.0 | 14952 | 2.7879 | 0.7416 |
| 0.0 | 85.0 | 15130 | 2.7926 | 0.7416 |
| 0.0 | 86.0 | 15308 | 2.7656 | 0.7472 |
| 0.0 | 87.0 | 15486 | 2.7336 | 0.7444 |
| 0.0 | 88.0 | 15664 | 2.7320 | 0.7444 |
| 0.0 | 89.0 | 15842 | 2.7402 | 0.7444 |
| 0.0 | 90.0 | 16020 | 2.7415 | 0.7444 |
| 0.0 | 91.0 | 16198 | 2.7406 | 0.7444 |
| 0.0 | 92.0 | 16376 | 2.7327 | 0.7444 |
| 0.0 | 93.0 | 16554 | 2.4082 | 0.7781 |
| 0.0 | 94.0 | 16732 | 2.4077 | 0.7753 |
| 0.0 | 95.0 | 16910 | 2.4185 | 0.7781 |
| 0.0 | 96.0 | 17088 | 2.6096 | 0.7528 |
| 0.0 | 97.0 | 17266 | 2.5907 | 0.7669 |
| 0.0 | 98.0 | 17444 | 2.6030 | 0.7556 |
| 0.0 | 99.0 | 17622 | 2.6081 | 0.7528 |
| 0.0 | 100.0 | 17800 | 2.6081 | 0.7528 |
Framework versions
- Transformers 4.27.3
- Pytorch 1.13.1+cu116
- Datasets 2.10.1
- Tokenizers 0.13.2
