<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
BERiT_2000_custom_architecture_40_epochs_ls_.1
This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 6.0606
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 40
- label_smoothing_factor: 0.1
Training results
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
15.8251 | 0.19 | 500 | 8.3567 |
7.8217 | 0.39 | 1000 | 7.2693 |
7.2486 | 0.58 | 1500 | 7.0533 |
7.0209 | 0.77 | 2000 | 6.9330 |
6.9572 | 0.97 | 2500 | 6.9266 |
6.897 | 1.16 | 3000 | 6.7520 |
6.8273 | 1.36 | 3500 | 6.6686 |
6.7692 | 1.55 | 4000 | 6.5613 |
6.7269 | 1.74 | 4500 | 6.5789 |
6.6783 | 1.94 | 5000 | 6.5036 |
6.6792 | 2.13 | 5500 | 6.5714 |
6.6638 | 2.32 | 6000 | 6.4733 |
6.5913 | 2.52 | 6500 | 6.4967 |
6.634 | 2.71 | 7000 | 6.4530 |
6.5831 | 2.9 | 7500 | 6.4763 |
6.556 | 3.1 | 8000 | 6.4616 |
6.5676 | 3.29 | 8500 | 6.4449 |
6.5447 | 3.49 | 9000 | 6.4795 |
6.5531 | 3.68 | 9500 | 6.4253 |
6.5356 | 3.87 | 10000 | 6.4508 |
6.5677 | 4.07 | 10500 | 6.4002 |
6.5035 | 4.26 | 11000 | 6.3985 |
6.5112 | 4.45 | 11500 | 6.4798 |
6.5038 | 4.65 | 12000 | 6.4138 |
6.504 | 4.84 | 12500 | 6.4381 |
6.4993 | 5.03 | 13000 | 6.4241 |
6.4853 | 5.23 | 13500 | 6.4275 |
6.4881 | 5.42 | 14000 | 6.3979 |
6.4882 | 5.62 | 14500 | 6.4468 |
6.4728 | 5.81 | 15000 | 6.4319 |
6.4853 | 6.0 | 15500 | 6.4124 |
6.4764 | 6.2 | 16000 | 6.4013 |
6.4638 | 6.39 | 16500 | 6.3955 |
6.4727 | 6.58 | 17000 | 6.4140 |
6.4595 | 6.78 | 17500 | 6.4229 |
6.4273 | 6.97 | 18000 | 6.3758 |
6.4594 | 7.16 | 18500 | 6.3889 |
6.4457 | 7.36 | 19000 | 6.3175 |
6.4538 | 7.55 | 19500 | 6.3748 |
6.453 | 7.75 | 20000 | 6.3782 |
6.4662 | 7.94 | 20500 | 6.3953 |
6.437 | 8.13 | 21000 | 6.4125 |
6.4342 | 8.33 | 21500 | 6.3641 |
6.4424 | 8.52 | 22000 | 6.3911 |
6.4035 | 8.71 | 22500 | 6.4061 |
6.44 | 8.91 | 23000 | 6.3751 |
6.4206 | 9.1 | 23500 | 6.4066 |
6.4297 | 9.3 | 24000 | 6.3342 |
6.408 | 9.49 | 24500 | 6.3508 |
6.4026 | 9.68 | 25000 | 6.3609 |
6.4262 | 9.88 | 25500 | 6.4123 |
6.4311 | 10.07 | 26000 | 6.3867 |
6.3788 | 10.26 | 26500 | 6.3724 |
6.3943 | 10.46 | 27000 | 6.4175 |
6.3867 | 10.65 | 27500 | 6.3834 |
6.3868 | 10.84 | 28000 | 6.4073 |
6.4119 | 11.04 | 28500 | 6.3267 |
6.387 | 11.23 | 29000 | 6.4038 |
6.4092 | 11.43 | 29500 | 6.3426 |
6.3994 | 11.62 | 30000 | 6.3827 |
6.3868 | 11.81 | 30500 | 6.3681 |
6.3779 | 12.01 | 31000 | 6.3419 |
6.3427 | 12.2 | 31500 | 6.2775 |
6.3763 | 12.39 | 32000 | 6.4021 |
6.349 | 12.59 | 32500 | 6.3800 |
6.3538 | 12.78 | 33000 | 6.3369 |
6.3734 | 12.97 | 33500 | 6.3495 |
6.3857 | 13.17 | 34000 | 6.3715 |
6.3678 | 13.36 | 34500 | 6.3540 |
6.3345 | 13.56 | 35000 | 6.3504 |
6.3399 | 13.75 | 35500 | 6.3641 |
6.3383 | 13.94 | 36000 | 6.3556 |
6.3637 | 14.14 | 36500 | 6.2868 |
6.3478 | 14.33 | 37000 | 6.3285 |
6.3416 | 14.52 | 37500 | 6.3001 |
6.3513 | 14.72 | 38000 | 6.3484 |
6.3381 | 14.91 | 38500 | 6.3030 |
6.3244 | 15.1 | 39000 | 6.3705 |
6.3285 | 15.3 | 39500 | 6.3402 |
6.3448 | 15.49 | 40000 | 6.2991 |
6.3199 | 15.69 | 40500 | 6.3100 |
6.3195 | 15.88 | 41000 | 6.3203 |
6.3191 | 16.07 | 41500 | 6.3647 |
6.3228 | 16.27 | 42000 | 6.3181 |
6.3055 | 16.46 | 42500 | 6.2765 |
6.2817 | 16.65 | 43000 | 6.3116 |
6.3412 | 16.85 | 43500 | 6.3228 |
6.3002 | 17.04 | 44000 | 6.3100 |
6.289 | 17.23 | 44500 | 6.3308 |
6.3087 | 17.43 | 45000 | 6.3143 |
6.2945 | 17.62 | 45500 | 6.3081 |
6.3012 | 17.82 | 46000 | 6.3268 |
6.3008 | 18.01 | 46500 | 6.2792 |
6.2712 | 18.2 | 47000 | 6.2773 |
6.2488 | 18.4 | 47500 | 6.3212 |
6.2927 | 18.59 | 48000 | 6.3141 |
6.2865 | 18.78 | 48500 | 6.2275 |
6.2472 | 18.98 | 49000 | 6.2689 |
6.2617 | 19.17 | 49500 | 6.2390 |
6.2627 | 19.36 | 50000 | 6.2496 |
6.2916 | 19.56 | 50500 | 6.2473 |
6.2514 | 19.75 | 51000 | 6.2867 |
6.2501 | 19.95 | 51500 | 6.2353 |
6.2242 | 20.14 | 52000 | 6.2676 |
6.2465 | 20.33 | 52500 | 6.2274 |
6.2677 | 20.53 | 53000 | 6.2365 |
6.2534 | 20.72 | 53500 | 6.2161 |
6.2185 | 20.91 | 54000 | 6.2284 |
6.2171 | 21.11 | 54500 | 6.2475 |
6.2322 | 21.3 | 55000 | 6.2339 |
6.2359 | 21.49 | 55500 | 6.2272 |
6.1961 | 21.69 | 56000 | 6.2844 |
6.2254 | 21.88 | 56500 | 6.1721 |
6.242 | 22.08 | 57000 | 6.2173 |
6.2136 | 22.27 | 57500 | 6.2512 |
6.2053 | 22.46 | 58000 | 6.1929 |
6.2052 | 22.66 | 58500 | 6.2275 |
6.2022 | 22.85 | 59000 | 6.1908 |
6.2127 | 23.04 | 59500 | 6.1896 |
6.2163 | 23.24 | 60000 | 6.1702 |
6.187 | 23.43 | 60500 | 6.2002 |
6.2149 | 23.63 | 61000 | 6.2151 |
6.1867 | 23.82 | 61500 | 6.1795 |
6.1901 | 24.01 | 62000 | 6.1942 |
6.1901 | 24.21 | 62500 | 6.1266 |
6.1959 | 24.4 | 63000 | 6.1754 |
6.2138 | 24.59 | 63500 | 6.1405 |
6.1917 | 24.79 | 64000 | 6.1818 |
6.204 | 24.98 | 64500 | 6.2095 |
6.1947 | 25.17 | 65000 | 6.1928 |
6.1793 | 25.37 | 65500 | 6.1076 |
6.1571 | 25.56 | 66000 | 6.1632 |
6.1595 | 25.76 | 66500 | 6.1803 |
6.1632 | 25.95 | 67000 | 6.1188 |
6.1756 | 26.14 | 67500 | 6.1569 |
6.1947 | 26.34 | 68000 | 6.1281 |
6.1451 | 26.53 | 68500 | 6.1972 |
6.1418 | 26.72 | 69000 | 6.1418 |
6.1657 | 26.92 | 69500 | 6.1514 |
6.1751 | 27.11 | 70000 | 6.1513 |
6.1529 | 27.3 | 70500 | 6.1319 |
6.155 | 27.5 | 71000 | 6.1515 |
6.1537 | 27.69 | 71500 | 6.1271 |
6.1372 | 27.89 | 72000 | 6.0880 |
6.1475 | 28.08 | 72500 | 6.1339 |
6.1254 | 28.27 | 73000 | 6.1503 |
6.1125 | 28.47 | 73500 | 6.1292 |
6.1359 | 28.66 | 74000 | 6.1513 |
6.1346 | 28.85 | 74500 | 6.1083 |
6.1411 | 29.05 | 75000 | 6.1005 |
6.147 | 29.24 | 75500 | 6.1451 |
6.1263 | 29.43 | 76000 | 6.1206 |
6.1211 | 29.63 | 76500 | 6.1363 |
6.0854 | 29.82 | 77000 | 6.1753 |
6.132 | 30.02 | 77500 | 6.1525 |
6.127 | 30.21 | 78000 | 6.1390 |
6.151 | 30.4 | 78500 | 6.1003 |
6.1272 | 30.6 | 79000 | 6.1126 |
6.0856 | 30.79 | 79500 | 6.1034 |
6.0806 | 30.98 | 80000 | 6.0432 |
6.1221 | 31.18 | 80500 | 6.1355 |
6.0959 | 31.37 | 81000 | 6.1013 |
6.1166 | 31.56 | 81500 | 6.1264 |
6.1337 | 31.76 | 82000 | 6.1278 |
6.0929 | 31.95 | 82500 | 6.0627 |
6.099 | 32.15 | 83000 | 6.0922 |
6.0934 | 32.34 | 83500 | 6.0839 |
6.1255 | 32.53 | 84000 | 6.1186 |
6.084 | 32.73 | 84500 | 6.0857 |
6.1205 | 32.92 | 85000 | 6.0830 |
6.0857 | 33.11 | 85500 | 6.1101 |
6.1016 | 33.31 | 86000 | 6.0707 |
6.1124 | 33.5 | 86500 | 6.1193 |
6.0906 | 33.69 | 87000 | 6.0991 |
6.1113 | 33.89 | 87500 | 6.1047 |
6.0811 | 34.08 | 88000 | 6.0694 |
6.0966 | 34.28 | 88500 | 6.0595 |
6.0931 | 34.47 | 89000 | 6.1141 |
6.1163 | 34.66 | 89500 | 6.0787 |
6.0874 | 34.86 | 90000 | 6.1036 |
6.0872 | 35.05 | 90500 | 6.1053 |
6.0804 | 35.24 | 91000 | 6.0759 |
6.0955 | 35.44 | 91500 | 6.0604 |
6.1155 | 35.63 | 92000 | 6.1016 |
6.0789 | 35.82 | 92500 | 6.0678 |
6.0605 | 36.02 | 93000 | 6.0467 |
6.0832 | 36.21 | 93500 | 6.0742 |
6.0742 | 36.41 | 94000 | 6.0379 |
6.0804 | 36.6 | 94500 | 6.0870 |
6.0629 | 36.79 | 95000 | 6.0339 |
6.0676 | 36.99 | 95500 | 6.0635 |
6.0838 | 37.18 | 96000 | 6.1115 |
6.0696 | 37.37 | 96500 | 6.1257 |
6.0761 | 37.57 | 97000 | 6.0905 |
6.073 | 37.76 | 97500 | 6.0920 |
6.059 | 37.96 | 98000 | 6.0508 |
6.0628 | 38.15 | 98500 | 6.0818 |
6.0383 | 38.34 | 99000 | 6.1400 |
6.0753 | 38.54 | 99500 | 6.0810 |
6.0641 | 38.73 | 100000 | 6.1257 |
6.0648 | 38.92 | 100500 | 6.0536 |
6.0545 | 39.12 | 101000 | 6.1098 |
6.0643 | 39.31 | 101500 | 6.0630 |
6.0521 | 39.5 | 102000 | 6.0639 |
6.0633 | 39.7 | 102500 | 6.0430 |
6.0464 | 39.89 | 103000 | 6.0606 |
Framework versions
- Transformers 4.24.0
- Pytorch 1.12.1+cu113
- Datasets 2.7.0
- Tokenizers 0.13.2