<!-- This model card has been generated automatically according to the information Keras had access to. You should probably proofread and complete it, then remove this comment. -->
Heem/distilroberta-finetuned-wtner
This model is a fine-tuned version of distilroberta-base on an unknown dataset. It achieves the following results on the evaluation set:
- Train Loss: 0.0055
- Validation Loss: 0.4521
- Train Precision: 0.7410
- Train Recall: 0.8122
- Train F1: 0.775
- Train Accuracy: 0.9382
- Epoch: 69
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 2030, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.01}
- training_precision: float32
Training results
Train Loss | Validation Loss | Train Precision | Train Recall | Train F1 | Train Accuracy | Epoch |
---|---|---|---|---|---|---|
1.3579 | 0.8909 | 0.0 | 0.0 | 0.0 | 0.7744 | 0 |
0.7332 | 0.6231 | 0.3526 | 0.2926 | 0.3198 | 0.8256 | 1 |
0.5037 | 0.4471 | 0.3927 | 0.3755 | 0.3839 | 0.8575 | 2 |
0.3675 | 0.3776 | 0.484 | 0.5284 | 0.5052 | 0.8855 | 3 |
0.2890 | 0.3519 | 0.5149 | 0.6026 | 0.5553 | 0.9039 | 4 |
0.2367 | 0.3317 | 0.5820 | 0.6507 | 0.6144 | 0.9150 | 5 |
0.1942 | 0.2970 | 0.6220 | 0.6900 | 0.6542 | 0.9237 | 6 |
0.1599 | 0.3040 | 0.6375 | 0.6681 | 0.6525 | 0.9217 | 7 |
0.1281 | 0.3037 | 0.6774 | 0.7336 | 0.7044 | 0.9304 | 8 |
0.1097 | 0.3127 | 0.708 | 0.7729 | 0.7390 | 0.9309 | 9 |
0.0915 | 0.3114 | 0.6836 | 0.7642 | 0.7216 | 0.9290 | 10 |
0.0765 | 0.3190 | 0.7072 | 0.8122 | 0.7561 | 0.9372 | 11 |
0.0665 | 0.3169 | 0.7154 | 0.7904 | 0.7510 | 0.9353 | 12 |
0.0543 | 0.3251 | 0.7059 | 0.7860 | 0.7438 | 0.9329 | 13 |
0.0472 | 0.3307 | 0.7181 | 0.8122 | 0.7623 | 0.9357 | 14 |
0.0427 | 0.3639 | 0.7148 | 0.7991 | 0.7546 | 0.9357 | 15 |
0.0380 | 0.3373 | 0.7373 | 0.8210 | 0.7769 | 0.9377 | 16 |
0.0380 | 0.3422 | 0.7449 | 0.8035 | 0.7731 | 0.9372 | 17 |
0.0304 | 0.3455 | 0.7530 | 0.8122 | 0.7815 | 0.9386 | 18 |
0.0271 | 0.3584 | 0.7294 | 0.8122 | 0.7686 | 0.9377 | 19 |
0.0249 | 0.3661 | 0.7291 | 0.7991 | 0.7625 | 0.9377 | 20 |
0.0205 | 0.3683 | 0.7352 | 0.8122 | 0.7718 | 0.9391 | 21 |
0.0212 | 0.3855 | 0.7331 | 0.8035 | 0.7667 | 0.9382 | 22 |
0.0188 | 0.3814 | 0.7419 | 0.8035 | 0.7715 | 0.9391 | 23 |
0.0189 | 0.3889 | 0.7352 | 0.8122 | 0.7718 | 0.9357 | 24 |
0.0161 | 0.3913 | 0.7379 | 0.7991 | 0.7673 | 0.9382 | 25 |
0.0154 | 0.3872 | 0.7470 | 0.8122 | 0.7782 | 0.9406 | 26 |
0.0144 | 0.3934 | 0.7326 | 0.8253 | 0.7762 | 0.9401 | 27 |
0.0154 | 0.4167 | 0.7255 | 0.8079 | 0.7645 | 0.9343 | 28 |
0.0135 | 0.3976 | 0.7341 | 0.8079 | 0.7692 | 0.9362 | 29 |
0.0119 | 0.4118 | 0.7510 | 0.8297 | 0.7884 | 0.9382 | 30 |
0.0103 | 0.4112 | 0.7323 | 0.8122 | 0.7702 | 0.9372 | 31 |
0.0103 | 0.4172 | 0.7362 | 0.8166 | 0.7743 | 0.9382 | 32 |
0.0111 | 0.4157 | 0.7283 | 0.8079 | 0.7660 | 0.9382 | 33 |
0.0103 | 0.4152 | 0.7262 | 0.7991 | 0.7609 | 0.9372 | 34 |
0.0117 | 0.4090 | 0.7188 | 0.8035 | 0.7588 | 0.9377 | 35 |
0.0098 | 0.4268 | 0.7302 | 0.8035 | 0.7651 | 0.9367 | 36 |
0.0082 | 0.4354 | 0.7233 | 0.7991 | 0.7593 | 0.9362 | 37 |
0.0096 | 0.4298 | 0.7154 | 0.7904 | 0.7510 | 0.9357 | 38 |
0.0093 | 0.4294 | 0.7273 | 0.8035 | 0.7635 | 0.9362 | 39 |
0.0084 | 0.4266 | 0.7298 | 0.7904 | 0.7589 | 0.9348 | 40 |
0.0076 | 0.4230 | 0.7251 | 0.7948 | 0.7583 | 0.9357 | 41 |
0.0068 | 0.4243 | 0.7075 | 0.7817 | 0.7427 | 0.9329 | 42 |
0.0080 | 0.4379 | 0.7137 | 0.7729 | 0.7421 | 0.9338 | 43 |
0.0067 | 0.4361 | 0.7302 | 0.8035 | 0.7651 | 0.9362 | 44 |
0.0066 | 0.4377 | 0.7341 | 0.8079 | 0.7692 | 0.9367 | 45 |
0.0056 | 0.4357 | 0.7222 | 0.7948 | 0.7568 | 0.9362 | 46 |
0.0060 | 0.4393 | 0.7205 | 0.7991 | 0.7578 | 0.9362 | 47 |
0.0060 | 0.4429 | 0.7194 | 0.7948 | 0.7552 | 0.9357 | 48 |
0.0054 | 0.4416 | 0.7312 | 0.8079 | 0.7676 | 0.9367 | 49 |
0.0060 | 0.4413 | 0.7188 | 0.8035 | 0.7588 | 0.9362 | 50 |
0.0058 | 0.4381 | 0.7344 | 0.8210 | 0.7753 | 0.9377 | 51 |
0.0063 | 0.4388 | 0.7309 | 0.7948 | 0.7615 | 0.9377 | 52 |
0.0057 | 0.4402 | 0.7412 | 0.8253 | 0.7810 | 0.9382 | 53 |
0.0052 | 0.4381 | 0.7362 | 0.8166 | 0.7743 | 0.9377 | 54 |
0.0049 | 0.4407 | 0.7362 | 0.8166 | 0.7743 | 0.9377 | 55 |
0.0050 | 0.4394 | 0.7490 | 0.8210 | 0.7833 | 0.9386 | 56 |
0.0047 | 0.4481 | 0.7460 | 0.8210 | 0.7817 | 0.9382 | 57 |
0.0052 | 0.4544 | 0.748 | 0.8166 | 0.7808 | 0.9367 | 58 |
0.0049 | 0.4501 | 0.7430 | 0.8079 | 0.7741 | 0.9362 | 59 |
0.0050 | 0.4504 | 0.744 | 0.8122 | 0.7766 | 0.9367 | 60 |
0.0047 | 0.4517 | 0.7312 | 0.8079 | 0.7676 | 0.9372 | 61 |
0.0049 | 0.4526 | 0.7450 | 0.8166 | 0.7792 | 0.9382 | 62 |
0.0049 | 0.4534 | 0.7490 | 0.8210 | 0.7833 | 0.9386 | 63 |
0.0056 | 0.4543 | 0.748 | 0.8166 | 0.7808 | 0.9386 | 64 |
0.0044 | 0.4522 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 65 |
0.0047 | 0.4522 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 66 |
0.0050 | 0.4521 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 67 |
0.0049 | 0.4521 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 68 |
0.0055 | 0.4521 | 0.7410 | 0.8122 | 0.775 | 0.9382 | 69 |
Framework versions
- Transformers 4.20.1
- TensorFlow 2.9.1
- Datasets 2.4.0
- Tokenizers 0.12.1