<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
tiny-mlm-glue-cola-custom-tokenizer-target-glue-mnli
This model is a fine-tuned version of muhtasham/tiny-mlm-glue-cola-custom-tokenizer on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.8453
- Accuracy: 0.6219
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 200
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 1.0867 | 0.04 | 500 | 1.0660 | 0.4244 |
| 1.0666 | 0.08 | 1000 | 1.0513 | 0.4444 |
| 1.0546 | 0.12 | 1500 | 1.0457 | 0.4512 |
| 1.049 | 0.16 | 2000 | 1.0372 | 0.4629 |
| 1.0419 | 0.2 | 2500 | 1.0297 | 0.4678 |
| 1.0351 | 0.24 | 3000 | 1.0151 | 0.4927 |
| 1.0061 | 0.29 | 3500 | 0.9871 | 0.5226 |
| 0.9924 | 0.33 | 4000 | 0.9772 | 0.5319 |
| 0.9733 | 0.37 | 4500 | 0.9672 | 0.5387 |
| 0.9704 | 0.41 | 5000 | 0.9534 | 0.5461 |
| 0.9603 | 0.45 | 5500 | 0.9442 | 0.5535 |
| 0.9572 | 0.49 | 6000 | 0.9423 | 0.5526 |
| 0.9537 | 0.53 | 6500 | 0.9302 | 0.5634 |
| 0.9487 | 0.57 | 7000 | 0.9320 | 0.5640 |
| 0.9413 | 0.61 | 7500 | 0.9145 | 0.5737 |
| 0.9348 | 0.65 | 8000 | 0.9110 | 0.5728 |
| 0.9305 | 0.69 | 8500 | 0.9089 | 0.5793 |
| 0.9244 | 0.73 | 9000 | 0.8992 | 0.5836 |
| 0.9242 | 0.77 | 9500 | 0.8970 | 0.5835 |
| 0.915 | 0.81 | 10000 | 0.8926 | 0.5851 |
| 0.9142 | 0.86 | 10500 | 0.8912 | 0.5901 |
| 0.9129 | 0.9 | 11000 | 0.8825 | 0.5965 |
| 0.9049 | 0.94 | 11500 | 0.8792 | 0.5998 |
| 0.9109 | 0.98 | 12000 | 0.8785 | 0.5977 |
| 0.8934 | 1.02 | 12500 | 0.8775 | 0.6016 |
| 0.8901 | 1.06 | 13000 | 0.8760 | 0.5989 |
| 0.8905 | 1.1 | 13500 | 0.8650 | 0.6050 |
| 0.8774 | 1.14 | 14000 | 0.8676 | 0.6051 |
| 0.8826 | 1.18 | 14500 | 0.8619 | 0.6089 |
| 0.8762 | 1.22 | 15000 | 0.8715 | 0.6030 |
| 0.8787 | 1.26 | 15500 | 0.8623 | 0.6120 |
| 0.8732 | 1.3 | 16000 | 0.8667 | 0.6092 |
| 0.8765 | 1.34 | 16500 | 0.8608 | 0.6092 |
| 0.8841 | 1.39 | 17000 | 0.8572 | 0.6128 |
| 0.877 | 1.43 | 17500 | 0.8604 | 0.6163 |
| 0.8724 | 1.47 | 18000 | 0.8538 | 0.6137 |
| 0.8712 | 1.51 | 18500 | 0.8496 | 0.6185 |
| 0.8735 | 1.55 | 19000 | 0.8515 | 0.6186 |
| 0.8705 | 1.59 | 19500 | 0.8471 | 0.6230 |
| 0.8676 | 1.63 | 20000 | 0.8476 | 0.6168 |
| 0.8655 | 1.67 | 20500 | 0.8503 | 0.6210 |
| 0.869 | 1.71 | 21000 | 0.8453 | 0.6219 |
Framework versions
- Transformers 4.26.0.dev0
- Pytorch 1.13.0+cu116
- Datasets 2.8.1.dev0
- Tokenizers 0.13.2
