<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
q05_kaggle_distilbert_inverted_weights
This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.0977
- Accuracy: 0.8581
- F1: 0.3892
- Recall: 0.4038
- Precision: 0.3758
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 0
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- label_smoothing_factor: 0.1
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Recall | Precision |
|---|---|---|---|---|---|---|---|
| 0.2883 | 0.86 | 50 | 0.1095 | 0.8041 | 0.3035 | 0.3072 | 0.3137 |
| 0.124 | 1.72 | 100 | 0.1317 | 0.8497 | 0.3806 | 0.3933 | 0.3687 |
| 0.1062 | 2.59 | 150 | 0.1585 | 0.8361 | 0.3900 | 0.4294 | 0.3674 |
| 0.0888 | 3.45 | 200 | 0.1000 | 0.8328 | 0.3500 | 0.3531 | 0.3505 |
| 0.0789 | 4.31 | 250 | 0.1004 | 0.8395 | 0.3555 | 0.3573 | 0.3587 |
| 0.0649 | 5.17 | 300 | 0.0977 | 0.8581 | 0.3892 | 0.4038 | 0.3758 |
| 0.0526 | 6.03 | 350 | 0.1649 | 0.8615 | 0.3985 | 0.4222 | 0.3794 |
| 0.0384 | 6.9 | 400 | 0.1455 | 0.8733 | 0.4581 | 0.4546 | 0.5005 |
| 0.0351 | 7.76 | 450 | 0.1883 | 0.8767 | 0.5082 | 0.4999 | 0.5376 |
| 0.0344 | 8.62 | 500 | 0.2364 | 0.8733 | 0.5062 | 0.5104 | 0.5179 |
| 0.024 | 9.48 | 550 | 0.1847 | 0.8767 | 0.5483 | 0.5109 | 0.6784 |
Framework versions
- Transformers 4.28.1
- Pytorch 2.0.0+cu118
- Datasets 2.12.0
- Tokenizers 0.13.3
