<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
distilbert-base-uncased-finetuned
This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 7.2813
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 8.6309 | 1.0 | 76 | 7.4774 |
| 7.0806 | 2.0 | 152 | 6.9937 |
| 6.6842 | 3.0 | 228 | 6.9314 |
| 6.4592 | 4.0 | 304 | 6.9088 |
| 6.2936 | 5.0 | 380 | 6.9135 |
| 6.1301 | 6.0 | 456 | 6.9018 |
| 5.9878 | 7.0 | 532 | 6.8865 |
| 5.8071 | 8.0 | 608 | 6.8926 |
| 5.6372 | 9.0 | 684 | 6.8750 |
| 5.4791 | 10.0 | 760 | 6.9394 |
| 5.3365 | 11.0 | 836 | 6.9594 |
| 5.2117 | 12.0 | 912 | 6.9962 |
| 5.0887 | 13.0 | 988 | 7.0570 |
| 4.9288 | 14.0 | 1064 | 7.0549 |
| 4.8169 | 15.0 | 1140 | 7.0971 |
| 4.7008 | 16.0 | 1216 | 7.1439 |
| 4.6149 | 17.0 | 1292 | 7.1320 |
| 4.487 | 18.0 | 1368 | 7.1577 |
| 4.364 | 19.0 | 1444 | 7.1712 |
| 4.3208 | 20.0 | 1520 | 7.1959 |
| 4.2492 | 21.0 | 1596 | 7.2136 |
| 4.1423 | 22.0 | 1672 | 7.2304 |
| 4.0873 | 23.0 | 1748 | 7.2526 |
| 4.0261 | 24.0 | 1824 | 7.2681 |
| 3.9598 | 25.0 | 1900 | 7.2715 |
| 3.9562 | 26.0 | 1976 | 7.2648 |
| 3.8951 | 27.0 | 2052 | 7.2665 |
| 3.8772 | 28.0 | 2128 | 7.2781 |
| 3.8403 | 29.0 | 2204 | 7.2801 |
| 3.8275 | 30.0 | 2280 | 7.2813 |
Framework versions
- Transformers 4.21.3
- Pytorch 1.12.1
- Datasets 2.4.0
- Tokenizers 0.12.1
