<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
BERiT_2000_custom_architecture_3
This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 5.6575
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 40
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 16.5165 | 0.19 | 500 | 8.9072 |
| 8.208 | 0.39 | 1000 | 7.5024 |
| 7.3849 | 0.58 | 1500 | 7.1180 |
| 7.0298 | 0.77 | 2000 | 6.8964 |
| 6.9022 | 0.97 | 2500 | 6.7857 |
| 6.7756 | 1.16 | 3000 | 6.5215 |
| 6.6462 | 1.36 | 3500 | 6.4494 |
| 6.5787 | 1.55 | 4000 | 6.3246 |
| 6.5193 | 1.74 | 4500 | 6.3231 |
| 6.4626 | 1.94 | 5000 | 6.2818 |
| 6.474 | 2.13 | 5500 | 6.3444 |
| 6.4314 | 2.32 | 6000 | 6.2374 |
| 6.3658 | 2.52 | 6500 | 6.2517 |
| 6.4031 | 2.71 | 7000 | 6.2055 |
| 6.3549 | 2.9 | 7500 | 6.2022 |
| 6.3202 | 3.1 | 8000 | 6.2163 |
| 6.3294 | 3.29 | 8500 | 6.2001 |
| 6.2981 | 3.49 | 9000 | 6.1819 |
| 6.3281 | 3.68 | 9500 | 6.1564 |
| 6.2914 | 3.87 | 10000 | 6.2122 |
| 6.3275 | 4.07 | 10500 | 6.1463 |
| 6.2637 | 4.26 | 11000 | 6.1404 |
| 6.2641 | 4.45 | 11500 | 6.2438 |
| 6.2557 | 4.65 | 12000 | 6.1504 |
| 6.2541 | 4.84 | 12500 | 6.1816 |
| 6.2465 | 5.03 | 13000 | 6.1646 |
| 6.2436 | 5.23 | 13500 | 6.1698 |
| 6.2461 | 5.42 | 14000 | 6.1665 |
| 6.2304 | 5.62 | 14500 | 6.1873 |
| 6.2235 | 5.81 | 15000 | 6.1555 |
| 6.2262 | 6.0 | 15500 | 6.1128 |
| 6.2238 | 6.2 | 16000 | 6.1545 |
| 6.2127 | 6.39 | 16500 | 6.1131 |
| 6.221 | 6.58 | 17000 | 6.1513 |
| 6.1974 | 6.78 | 17500 | 6.1712 |
| 6.175 | 6.97 | 18000 | 6.1073 |
| 6.2042 | 7.16 | 18500 | 6.1176 |
| 6.1898 | 7.36 | 19000 | 6.0470 |
| 6.1961 | 7.55 | 19500 | 6.1011 |
| 6.1883 | 7.75 | 20000 | 6.1064 |
| 6.2171 | 7.94 | 20500 | 6.1299 |
| 6.175 | 8.13 | 21000 | 6.1313 |
| 6.1757 | 8.33 | 21500 | 6.0899 |
| 6.1776 | 8.52 | 22000 | 6.1196 |
| 6.1377 | 8.71 | 22500 | 6.1554 |
| 6.1688 | 8.91 | 23000 | 6.1037 |
| 6.1555 | 9.1 | 23500 | 6.1622 |
| 6.1665 | 9.3 | 24000 | 6.0622 |
| 6.144 | 9.49 | 24500 | 6.0763 |
| 6.1394 | 9.68 | 25000 | 6.0803 |
| 6.1731 | 9.88 | 25500 | 6.1243 |
| 6.1655 | 10.07 | 26000 | 6.0929 |
| 6.1028 | 10.26 | 26500 | 6.1178 |
| 6.1145 | 10.46 | 27000 | 6.1426 |
| 6.1153 | 10.65 | 27500 | 6.1156 |
| 6.1274 | 10.84 | 28000 | 6.0922 |
| 6.1441 | 11.04 | 28500 | 6.0556 |
| 6.1179 | 11.23 | 29000 | 6.1316 |
| 6.1379 | 11.43 | 29500 | 6.0560 |
| 6.1273 | 11.62 | 30000 | 6.1321 |
| 6.1104 | 11.81 | 30500 | 6.1229 |
| 6.1156 | 12.01 | 31000 | 6.0803 |
| 6.0711 | 12.2 | 31500 | 6.0110 |
| 6.1132 | 12.39 | 32000 | 6.1489 |
| 6.065 | 12.59 | 32500 | 6.1082 |
| 6.0774 | 12.78 | 33000 | 6.0590 |
| 6.096 | 12.97 | 33500 | 6.0611 |
| 6.1172 | 13.17 | 34000 | 6.0857 |
| 6.0845 | 13.36 | 34500 | 6.0799 |
| 6.0551 | 13.56 | 35000 | 6.0768 |
| 6.0593 | 13.75 | 35500 | 6.0880 |
| 6.0605 | 13.94 | 36000 | 6.0715 |
| 6.0849 | 14.14 | 36500 | 5.9769 |
| 6.0739 | 14.33 | 37000 | 6.0450 |
| 6.0721 | 14.52 | 37500 | 6.0144 |
| 6.0778 | 14.72 | 38000 | 6.0817 |
| 6.067 | 14.91 | 38500 | 6.0142 |
| 6.0456 | 15.1 | 39000 | 6.1092 |
| 6.0624 | 15.3 | 39500 | 6.0543 |
| 6.0556 | 15.49 | 40000 | 6.0204 |
| 6.0358 | 15.69 | 40500 | 6.0146 |
| 6.0397 | 15.88 | 41000 | 6.0312 |
| 6.0352 | 16.07 | 41500 | 6.0761 |
| 6.0356 | 16.27 | 42000 | 6.0177 |
| 6.0149 | 16.46 | 42500 | 6.0044 |
| 5.9803 | 16.65 | 43000 | 6.0192 |
| 6.0615 | 16.85 | 43500 | 6.0227 |
| 6.0029 | 17.04 | 44000 | 6.0205 |
| 6.0005 | 17.23 | 44500 | 6.0298 |
| 6.0087 | 17.43 | 45000 | 5.9892 |
| 5.9895 | 17.62 | 45500 | 5.9715 |
| 6.0123 | 17.82 | 46000 | 6.0088 |
| 6.0015 | 18.01 | 46500 | 5.9670 |
| 5.9764 | 18.2 | 47000 | 5.9593 |
| 5.9399 | 18.4 | 47500 | 6.0001 |
| 5.9928 | 18.59 | 48000 | 5.9966 |
| 5.9823 | 18.78 | 48500 | 5.8836 |
| 5.9442 | 18.98 | 49000 | 5.9294 |
| 5.9532 | 19.17 | 49500 | 5.9487 |
| 5.9551 | 19.36 | 50000 | 5.9434 |
| 5.996 | 19.56 | 50500 | 5.9254 |
| 5.9468 | 19.75 | 51000 | 5.9532 |
| 5.9349 | 19.95 | 51500 | 5.9212 |
| 5.9155 | 20.14 | 52000 | 5.9140 |
| 5.9382 | 20.33 | 52500 | 5.8989 |
| 5.9538 | 20.53 | 53000 | 5.9010 |
| 5.9466 | 20.72 | 53500 | 5.8780 |
| 5.9112 | 20.91 | 54000 | 5.8883 |
| 5.908 | 21.11 | 54500 | 5.9060 |
| 5.9228 | 21.3 | 55000 | 5.8949 |
| 5.9428 | 21.49 | 55500 | 5.8879 |
| 5.8808 | 21.69 | 56000 | 5.9383 |
| 5.9311 | 21.88 | 56500 | 5.8401 |
| 5.936 | 22.08 | 57000 | 5.9064 |
| 5.8951 | 22.27 | 57500 | 5.8957 |
| 5.8832 | 22.46 | 58000 | 5.8583 |
| 5.8919 | 22.66 | 58500 | 5.8893 |
| 5.8884 | 22.85 | 59000 | 5.8666 |
| 5.9072 | 23.04 | 59500 | 5.8368 |
| 5.8971 | 23.24 | 60000 | 5.8299 |
| 5.868 | 23.43 | 60500 | 5.8595 |
| 5.8967 | 23.63 | 61000 | 5.8722 |
| 5.8746 | 23.82 | 61500 | 5.8307 |
| 5.8731 | 24.01 | 62000 | 5.8595 |
| 5.8625 | 24.21 | 62500 | 5.7892 |
| 5.8877 | 24.4 | 63000 | 5.8079 |
| 5.9033 | 24.59 | 63500 | 5.7787 |
| 5.8676 | 24.79 | 64000 | 5.8450 |
| 5.889 | 24.98 | 64500 | 5.8286 |
| 5.8732 | 25.17 | 65000 | 5.8433 |
| 5.8684 | 25.37 | 65500 | 5.7564 |
| 5.8516 | 25.56 | 66000 | 5.8181 |
| 5.835 | 25.76 | 66500 | 5.8275 |
| 5.8523 | 25.95 | 67000 | 5.7860 |
| 5.8612 | 26.14 | 67500 | 5.8005 |
| 5.8715 | 26.34 | 68000 | 5.7788 |
| 5.8191 | 26.53 | 68500 | 5.8558 |
| 5.8286 | 26.72 | 69000 | 5.7973 |
| 5.8415 | 26.92 | 69500 | 5.7792 |
| 5.855 | 27.11 | 70000 | 5.8006 |
| 5.8384 | 27.3 | 70500 | 5.7673 |
| 5.825 | 27.5 | 71000 | 5.8130 |
| 5.8243 | 27.69 | 71500 | 5.7763 |
| 5.8242 | 27.89 | 72000 | 5.7433 |
| 5.8251 | 28.08 | 72500 | 5.7670 |
| 5.8022 | 28.27 | 73000 | 5.8067 |
| 5.8014 | 28.47 | 73500 | 5.7979 |
| 5.8013 | 28.66 | 74000 | 5.7940 |
| 5.8154 | 28.85 | 74500 | 5.7362 |
| 5.8046 | 29.05 | 75000 | 5.7319 |
| 5.8222 | 29.24 | 75500 | 5.7902 |
| 5.7801 | 29.43 | 76000 | 5.7563 |
| 5.7932 | 29.63 | 76500 | 5.7724 |
| 5.7543 | 29.82 | 77000 | 5.8041 |
| 5.7936 | 30.02 | 77500 | 5.8168 |
| 5.8053 | 30.21 | 78000 | 5.7699 |
| 5.8103 | 30.4 | 78500 | 5.7276 |
| 5.8019 | 30.6 | 79000 | 5.7498 |
| 5.7647 | 30.79 | 79500 | 5.7413 |
| 5.7424 | 30.98 | 80000 | 5.6823 |
| 5.8021 | 31.18 | 80500 | 5.7597 |
| 5.7717 | 31.37 | 81000 | 5.7509 |
| 5.7908 | 31.56 | 81500 | 5.7664 |
| 5.8212 | 31.76 | 82000 | 5.7693 |
| 5.7733 | 31.95 | 82500 | 5.6974 |
| 5.7672 | 32.15 | 83000 | 5.6966 |
| 5.7533 | 32.34 | 83500 | 5.7002 |
| 5.7898 | 32.53 | 84000 | 5.7604 |
| 5.7422 | 32.73 | 84500 | 5.7043 |
| 5.7864 | 32.92 | 85000 | 5.6966 |
| 5.7563 | 33.11 | 85500 | 5.7300 |
| 5.7747 | 33.31 | 86000 | 5.6817 |
| 5.7718 | 33.5 | 86500 | 5.7329 |
| 5.7416 | 33.69 | 87000 | 5.7174 |
| 5.7838 | 33.89 | 87500 | 5.7136 |
| 5.7499 | 34.08 | 88000 | 5.6524 |
| 5.7716 | 34.28 | 88500 | 5.6702 |
| 5.7486 | 34.47 | 89000 | 5.7338 |
| 5.7932 | 34.66 | 89500 | 5.6822 |
| 5.7593 | 34.86 | 90000 | 5.7193 |
| 5.759 | 35.05 | 90500 | 5.7241 |
| 5.749 | 35.24 | 91000 | 5.6964 |
| 5.7548 | 35.44 | 91500 | 5.6691 |
| 5.7843 | 35.63 | 92000 | 5.7158 |
| 5.7464 | 35.82 | 92500 | 5.6574 |
| 5.735 | 36.02 | 93000 | 5.6470 |
| 5.7466 | 36.21 | 93500 | 5.6833 |
| 5.74 | 36.41 | 94000 | 5.6346 |
| 5.7464 | 36.6 | 94500 | 5.6980 |
| 5.7194 | 36.79 | 95000 | 5.6459 |
| 5.7328 | 36.99 | 95500 | 5.6634 |
| 5.7392 | 37.18 | 96000 | 5.7234 |
| 5.7422 | 37.37 | 96500 | 5.7338 |
| 5.7469 | 37.57 | 97000 | 5.7001 |
| 5.74 | 37.76 | 97500 | 5.7040 |
| 5.7321 | 37.96 | 98000 | 5.6562 |
| 5.7153 | 38.15 | 98500 | 5.6962 |
| 5.7066 | 38.34 | 99000 | 5.7527 |
| 5.7465 | 38.54 | 99500 | 5.6827 |
| 5.7364 | 38.73 | 100000 | 5.7359 |
| 5.7342 | 38.92 | 100500 | 5.6403 |
| 5.7281 | 39.12 | 101000 | 5.7184 |
| 5.7213 | 39.31 | 101500 | 5.6506 |
| 5.7069 | 39.5 | 102000 | 5.6693 |
| 5.7109 | 39.7 | 102500 | 5.6412 |
| 5.7142 | 39.89 | 103000 | 5.6575 |
Framework versions
- Transformers 4.24.0
- Pytorch 1.12.1+cu113
- Datasets 2.6.1
- Tokenizers 0.13.2
