<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
bert-large-uncased-sst-2-16-13-smoothed
This model is a fine-tuned version of bert-large-uncased on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.6556
- Accuracy: 0.75
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 50
- num_epochs: 75
- label_smoothing_factor: 0.45
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| No log | 1.0 | 1 | 0.7309 | 0.5 |
| No log | 2.0 | 2 | 0.7304 | 0.5 |
| No log | 3.0 | 3 | 0.7294 | 0.5 |
| No log | 4.0 | 4 | 0.7280 | 0.5 |
| No log | 5.0 | 5 | 0.7261 | 0.5 |
| No log | 6.0 | 6 | 0.7237 | 0.5 |
| No log | 7.0 | 7 | 0.7212 | 0.5 |
| No log | 8.0 | 8 | 0.7186 | 0.5 |
| No log | 9.0 | 9 | 0.7160 | 0.5 |
| 0.7321 | 10.0 | 10 | 0.7135 | 0.5 |
| 0.7321 | 11.0 | 11 | 0.7113 | 0.5 |
| 0.7321 | 12.0 | 12 | 0.7091 | 0.5 |
| 0.7321 | 13.0 | 13 | 0.7070 | 0.5 |
| 0.7321 | 14.0 | 14 | 0.7051 | 0.5 |
| 0.7321 | 15.0 | 15 | 0.7030 | 0.4688 |
| 0.7321 | 16.0 | 16 | 0.7012 | 0.4688 |
| 0.7321 | 17.0 | 17 | 0.6991 | 0.5 |
| 0.7321 | 18.0 | 18 | 0.6970 | 0.4688 |
| 0.7321 | 19.0 | 19 | 0.6949 | 0.5 |
| 0.6849 | 20.0 | 20 | 0.6930 | 0.5 |
| 0.6849 | 21.0 | 21 | 0.6914 | 0.4688 |
| 0.6849 | 22.0 | 22 | 0.6903 | 0.4688 |
| 0.6849 | 23.0 | 23 | 0.6895 | 0.5312 |
| 0.6849 | 24.0 | 24 | 0.6890 | 0.4688 |
| 0.6849 | 25.0 | 25 | 0.6883 | 0.4688 |
| 0.6849 | 26.0 | 26 | 0.6881 | 0.4688 |
| 0.6849 | 27.0 | 27 | 0.6877 | 0.5 |
| 0.6849 | 28.0 | 28 | 0.6867 | 0.5312 |
| 0.6849 | 29.0 | 29 | 0.6856 | 0.625 |
| 0.6342 | 30.0 | 30 | 0.6838 | 0.625 |
| 0.6342 | 31.0 | 31 | 0.6815 | 0.625 |
| 0.6342 | 32.0 | 32 | 0.6794 | 0.625 |
| 0.6342 | 33.0 | 33 | 0.6766 | 0.6562 |
| 0.6342 | 34.0 | 34 | 0.6739 | 0.625 |
| 0.6342 | 35.0 | 35 | 0.6715 | 0.625 |
| 0.6342 | 36.0 | 36 | 0.6692 | 0.6562 |
| 0.6342 | 37.0 | 37 | 0.6668 | 0.6875 |
| 0.6342 | 38.0 | 38 | 0.6646 | 0.6875 |
| 0.6342 | 39.0 | 39 | 0.6633 | 0.7188 |
| 0.5794 | 40.0 | 40 | 0.6624 | 0.7188 |
| 0.5794 | 41.0 | 41 | 0.6612 | 0.7188 |
| 0.5794 | 42.0 | 42 | 0.6600 | 0.75 |
| 0.5794 | 43.0 | 43 | 0.6601 | 0.75 |
| 0.5794 | 44.0 | 44 | 0.6602 | 0.75 |
| 0.5794 | 45.0 | 45 | 0.6609 | 0.75 |
| 0.5794 | 46.0 | 46 | 0.6629 | 0.6875 |
| 0.5794 | 47.0 | 47 | 0.6647 | 0.6875 |
| 0.5794 | 48.0 | 48 | 0.6635 | 0.7188 |
| 0.5794 | 49.0 | 49 | 0.6626 | 0.75 |
| 0.5487 | 50.0 | 50 | 0.6548 | 0.75 |
| 0.5487 | 51.0 | 51 | 0.6497 | 0.75 |
| 0.5487 | 52.0 | 52 | 0.6488 | 0.75 |
| 0.5487 | 53.0 | 53 | 0.6507 | 0.6875 |
| 0.5487 | 54.0 | 54 | 0.6529 | 0.6875 |
| 0.5487 | 55.0 | 55 | 0.6555 | 0.7188 |
| 0.5487 | 56.0 | 56 | 0.6577 | 0.7188 |
| 0.5487 | 57.0 | 57 | 0.6586 | 0.7188 |
| 0.5487 | 58.0 | 58 | 0.6587 | 0.7188 |
| 0.5487 | 59.0 | 59 | 0.6585 | 0.7188 |
| 0.5401 | 60.0 | 60 | 0.6592 | 0.6875 |
| 0.5401 | 61.0 | 61 | 0.6611 | 0.6875 |
| 0.5401 | 62.0 | 62 | 0.6623 | 0.6875 |
| 0.5401 | 63.0 | 63 | 0.6618 | 0.6875 |
| 0.5401 | 64.0 | 64 | 0.6601 | 0.6875 |
| 0.5401 | 65.0 | 65 | 0.6584 | 0.6875 |
| 0.5401 | 66.0 | 66 | 0.6570 | 0.7188 |
| 0.5401 | 67.0 | 67 | 0.6562 | 0.7188 |
| 0.5401 | 68.0 | 68 | 0.6555 | 0.7188 |
| 0.5401 | 69.0 | 69 | 0.6553 | 0.75 |
| 0.5397 | 70.0 | 70 | 0.6553 | 0.75 |
| 0.5397 | 71.0 | 71 | 0.6552 | 0.75 |
| 0.5397 | 72.0 | 72 | 0.6553 | 0.75 |
| 0.5397 | 73.0 | 73 | 0.6554 | 0.75 |
| 0.5397 | 74.0 | 74 | 0.6555 | 0.75 |
| 0.5397 | 75.0 | 75 | 0.6556 | 0.75 |
Framework versions
- Transformers 4.32.0.dev0
- Pytorch 2.0.1+cu118
- Datasets 2.4.0
- Tokenizers 0.13.3
