<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
tiny-mlm-glue-mnli-target-glue-qqp
This model is a fine-tuned version of muhtasham/tiny-mlm-glue-mnli on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.4097
- Accuracy: 0.7989
- F1: 0.7715
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- num_epochs: 200
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
---|---|---|---|---|---|
0.5775 | 0.04 | 500 | 0.5143 | 0.7309 | 0.6804 |
0.5085 | 0.09 | 1000 | 0.4819 | 0.7521 | 0.7031 |
0.4956 | 0.13 | 1500 | 0.4903 | 0.7412 | 0.7150 |
0.4795 | 0.18 | 2000 | 0.4662 | 0.7565 | 0.7230 |
0.4723 | 0.22 | 2500 | 0.4550 | 0.7643 | 0.7288 |
0.4562 | 0.26 | 3000 | 0.4472 | 0.7700 | 0.7353 |
0.4568 | 0.31 | 3500 | 0.4453 | 0.7695 | 0.7378 |
0.4502 | 0.35 | 4000 | 0.4475 | 0.7680 | 0.7398 |
0.4504 | 0.4 | 4500 | 0.4412 | 0.7721 | 0.7435 |
0.4406 | 0.44 | 5000 | 0.4203 | 0.7893 | 0.7497 |
0.4289 | 0.48 | 5500 | 0.4252 | 0.7859 | 0.7521 |
0.429 | 0.53 | 6000 | 0.4063 | 0.8017 | 0.7584 |
0.4237 | 0.57 | 6500 | 0.4189 | 0.7908 | 0.7566 |
0.4257 | 0.62 | 7000 | 0.4035 | 0.8032 | 0.7603 |
0.4169 | 0.66 | 7500 | 0.4044 | 0.8059 | 0.7650 |
0.4306 | 0.7 | 8000 | 0.4104 | 0.7969 | 0.7617 |
0.4199 | 0.75 | 8500 | 0.3954 | 0.8086 | 0.7675 |
0.414 | 0.79 | 9000 | 0.4181 | 0.7907 | 0.7625 |
0.4117 | 0.84 | 9500 | 0.3887 | 0.8118 | 0.7679 |
0.4137 | 0.88 | 10000 | 0.3852 | 0.8166 | 0.7721 |
0.4087 | 0.92 | 10500 | 0.3832 | 0.8158 | 0.7709 |
0.4082 | 0.97 | 11000 | 0.3831 | 0.8190 | 0.7734 |
0.4052 | 1.01 | 11500 | 0.4005 | 0.8039 | 0.7715 |
0.3844 | 1.06 | 12000 | 0.3799 | 0.8193 | 0.7751 |
0.3951 | 1.1 | 12500 | 0.3833 | 0.8154 | 0.7771 |
0.3825 | 1.14 | 13000 | 0.3874 | 0.8155 | 0.7775 |
0.3721 | 1.19 | 13500 | 0.4097 | 0.7989 | 0.7715 |
Framework versions
- Transformers 4.26.0.dev0
- Pytorch 1.13.0+cu116
- Datasets 2.8.1.dev0
- Tokenizers 0.13.2