<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
BERiT_2000_custom_architecture_2
This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 5.9854
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 16.4316 | 0.19 | 500 | 9.0685 |
| 8.2958 | 0.39 | 1000 | 7.6483 |
| 7.4324 | 0.58 | 1500 | 7.1707 |
| 7.0054 | 0.77 | 2000 | 6.8592 |
| 6.8522 | 0.97 | 2500 | 6.7710 |
| 6.7538 | 1.16 | 3000 | 6.5845 |
| 6.634 | 1.36 | 3500 | 6.4525 |
| 6.5784 | 1.55 | 4000 | 6.3129 |
| 6.5135 | 1.74 | 4500 | 6.3312 |
| 6.4552 | 1.94 | 5000 | 6.2546 |
| 6.4685 | 2.13 | 5500 | 6.2857 |
| 6.4356 | 2.32 | 6000 | 6.2285 |
| 6.3566 | 2.52 | 6500 | 6.2295 |
| 6.394 | 2.71 | 7000 | 6.1790 |
| 6.3412 | 2.9 | 7500 | 6.1880 |
| 6.3115 | 3.1 | 8000 | 6.2130 |
| 6.3163 | 3.29 | 8500 | 6.1831 |
| 6.2978 | 3.49 | 9000 | 6.1945 |
| 6.3082 | 3.68 | 9500 | 6.1485 |
| 6.2729 | 3.87 | 10000 | 6.1752 |
| 6.307 | 4.07 | 10500 | 6.1331 |
| 6.2494 | 4.26 | 11000 | 6.1082 |
| 6.2523 | 4.45 | 11500 | 6.2110 |
| 6.2455 | 4.65 | 12000 | 6.1326 |
| 6.2399 | 4.84 | 12500 | 6.1779 |
| 6.2297 | 5.03 | 13000 | 6.1587 |
| 6.2374 | 5.23 | 13500 | 6.1458 |
| 6.2265 | 5.42 | 14000 | 6.1370 |
| 6.2222 | 5.62 | 14500 | 6.1511 |
| 6.2209 | 5.81 | 15000 | 6.1320 |
| 6.2146 | 6.0 | 15500 | 6.1124 |
| 6.214 | 6.2 | 16000 | 6.1439 |
| 6.1907 | 6.39 | 16500 | 6.0981 |
| 6.2119 | 6.58 | 17000 | 6.1465 |
| 6.1858 | 6.78 | 17500 | 6.1594 |
| 6.1552 | 6.97 | 18000 | 6.0742 |
| 6.1926 | 7.16 | 18500 | 6.1176 |
| 6.1813 | 7.36 | 19000 | 6.0107 |
| 6.1812 | 7.55 | 19500 | 6.0852 |
| 6.1852 | 7.75 | 20000 | 6.0845 |
| 6.1945 | 7.94 | 20500 | 6.1260 |
| 6.1542 | 8.13 | 21000 | 6.1032 |
| 6.1685 | 8.33 | 21500 | 6.0650 |
| 6.1619 | 8.52 | 22000 | 6.1028 |
| 6.1279 | 8.71 | 22500 | 6.1269 |
| 6.1575 | 8.91 | 23000 | 6.0793 |
| 6.1401 | 9.1 | 23500 | 6.1479 |
| 6.159 | 9.3 | 24000 | 6.0319 |
| 6.1227 | 9.49 | 24500 | 6.0677 |
| 6.1201 | 9.68 | 25000 | 6.0527 |
| 6.1473 | 9.88 | 25500 | 6.1305 |
| 6.1539 | 10.07 | 26000 | 6.1079 |
| 6.091 | 10.26 | 26500 | 6.1219 |
| 6.1015 | 10.46 | 27000 | 6.1317 |
| 6.1048 | 10.65 | 27500 | 6.1149 |
| 6.0955 | 10.84 | 28000 | 6.1216 |
| 6.129 | 11.04 | 28500 | 6.0427 |
| 6.1007 | 11.23 | 29000 | 6.1289 |
| 6.1266 | 11.43 | 29500 | 6.0564 |
| 6.1203 | 11.62 | 30000 | 6.1143 |
| 6.1038 | 11.81 | 30500 | 6.0957 |
| 6.0989 | 12.01 | 31000 | 6.0707 |
| 6.0571 | 12.2 | 31500 | 6.0013 |
| 6.1017 | 12.39 | 32000 | 6.1356 |
| 6.0649 | 12.59 | 32500 | 6.0981 |
| 6.0704 | 12.78 | 33000 | 6.0588 |
| 6.088 | 12.97 | 33500 | 6.0796 |
| 6.1112 | 13.17 | 34000 | 6.0809 |
| 6.0888 | 13.36 | 34500 | 6.0776 |
| 6.0482 | 13.56 | 35000 | 6.0710 |
| 6.0588 | 13.75 | 35500 | 6.0877 |
| 6.0517 | 13.94 | 36000 | 6.0650 |
| 6.0832 | 14.14 | 36500 | 5.9890 |
| 6.0655 | 14.33 | 37000 | 6.0445 |
| 6.0705 | 14.52 | 37500 | 6.0037 |
| 6.0789 | 14.72 | 38000 | 6.0777 |
| 6.0645 | 14.91 | 38500 | 6.0475 |
| 6.0347 | 15.1 | 39000 | 6.1148 |
| 6.0478 | 15.3 | 39500 | 6.0639 |
| 6.0638 | 15.49 | 40000 | 6.0373 |
| 6.0377 | 15.69 | 40500 | 6.0116 |
| 6.0402 | 15.88 | 41000 | 6.0483 |
| 6.0382 | 16.07 | 41500 | 6.1025 |
| 6.039 | 16.27 | 42000 | 6.0488 |
| 6.0232 | 16.46 | 42500 | 6.0219 |
| 5.9946 | 16.65 | 43000 | 6.0541 |
| 6.063 | 16.85 | 43500 | 6.0436 |
| 6.0141 | 17.04 | 44000 | 6.0609 |
| 6.0196 | 17.23 | 44500 | 6.0551 |
| 6.0331 | 17.43 | 45000 | 6.0576 |
| 6.0174 | 17.62 | 45500 | 6.0498 |
| 6.0366 | 17.82 | 46000 | 6.0782 |
| 6.0299 | 18.01 | 46500 | 6.0196 |
| 6.0009 | 18.2 | 47000 | 6.0262 |
| 5.9758 | 18.4 | 47500 | 6.0824 |
| 6.0285 | 18.59 | 48000 | 6.0799 |
| 6.025 | 18.78 | 48500 | 5.9511 |
| 5.9806 | 18.98 | 49000 | 6.0086 |
| 5.9915 | 19.17 | 49500 | 6.0089 |
| 5.9957 | 19.36 | 50000 | 6.0330 |
| 6.0311 | 19.56 | 50500 | 6.0083 |
| 5.995 | 19.75 | 51000 | 6.0394 |
| 6.0034 | 19.95 | 51500 | 5.9854 |
Framework versions
- Transformers 4.24.0
- Pytorch 1.12.1+cu113
- Datasets 2.6.1
- Tokenizers 0.13.2
