<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
Debertalarg_model_multichoice
This model is a fine-tuned version of VuongQuoc/Debertalarg_model_multichoice on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.6374
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-06
- train_batch_size: 2
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.8
- num_epochs: 2
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.1605 | 0.04 | 500 | 0.4608 |
| 0.9286 | 0.08 | 1000 | 0.4628 |
| 1.0432 | 0.12 | 1500 | 0.4584 |
| 0.9711 | 0.15 | 2000 | 0.4621 |
| 0.9773 | 0.19 | 2500 | 0.4663 |
| 0.8774 | 0.23 | 3000 | 0.4755 |
| 0.928 | 0.27 | 3500 | 0.4788 |
| 0.7664 | 0.31 | 4000 | 0.5324 |
| 0.8086 | 0.35 | 4500 | 0.5547 |
| 0.7734 | 0.39 | 5000 | 0.5291 |
| 0.6808 | 0.42 | 5500 | 0.6206 |
| 0.7154 | 0.46 | 6000 | 0.5475 |
| 0.5706 | 0.5 | 6500 | 0.6125 |
| 0.5246 | 0.54 | 7000 | 0.6268 |
| 0.5604 | 0.58 | 7500 | 0.6162 |
| 0.5964 | 0.62 | 8000 | 0.5577 |
| 0.5431 | 0.65 | 8500 | 0.6210 |
| 0.3767 | 0.69 | 9000 | 0.6689 |
| 1.3041 | 0.73 | 9500 | 0.4854 |
| 1.1143 | 0.77 | 10000 | 0.4706 |
| 1.0349 | 0.81 | 10500 | 0.4753 |
| 1.0821 | 0.85 | 11000 | 0.4792 |
| 1.1573 | 0.89 | 11500 | 0.4748 |
| 1.023 | 0.92 | 12000 | 0.5218 |
| 1.0442 | 0.96 | 12500 | 0.4865 |
| 1.0542 | 1.0 | 13000 | 0.4822 |
| 0.8776 | 1.04 | 13500 | 0.5088 |
| 0.8876 | 1.08 | 14000 | 0.5967 |
| 0.9596 | 1.12 | 14500 | 0.5348 |
| 0.9764 | 1.16 | 15000 | 0.6067 |
| 0.9227 | 1.19 | 15500 | 0.5799 |
| 1.0766 | 1.23 | 16000 | 0.6245 |
| 0.9904 | 1.27 | 16500 | 0.6092 |
| 0.9606 | 1.31 | 17000 | 0.4992 |
| 0.9603 | 1.35 | 17500 | 0.5810 |
| 0.9904 | 1.39 | 18000 | 0.5620 |
| 1.0295 | 1.43 | 18500 | 0.5608 |
| 1.0057 | 1.46 | 19000 | 0.5252 |
| 1.0016 | 1.5 | 19500 | 0.5896 |
| 0.9924 | 1.54 | 20000 | 0.5939 |
| 0.9913 | 1.58 | 20500 | 0.5290 |
| 0.9345 | 1.62 | 21000 | 0.6177 |
| 1.0221 | 1.66 | 21500 | 0.6169 |
| 0.9862 | 1.7 | 22000 | 0.7013 |
| 0.9735 | 1.73 | 22500 | 0.6552 |
| 0.9601 | 1.77 | 23000 | 0.6497 |
| 0.923 | 1.81 | 23500 | 0.6733 |
| 0.9139 | 1.85 | 24000 | 0.7223 |
| 0.9713 | 1.89 | 24500 | 0.6593 |
| 1.015 | 1.93 | 25000 | 0.6622 |
| 1.0154 | 1.96 | 25500 | 0.6374 |
Framework versions
- Transformers 4.30.2
- Pytorch 2.0.0
- Datasets 2.1.0
- Tokenizers 0.13.3
