<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
BERiT_2000_custom_architecture_40_epochs_ls_.2
This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 6.3120
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 40
- label_smoothing_factor: 0.2
Training results
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
15.998 | 0.19 | 500 | 8.5537 |
7.8818 | 0.39 | 1000 | 7.3646 |
7.2781 | 0.58 | 1500 | 7.1307 |
7.1073 | 0.77 | 2000 | 7.0462 |
7.0749 | 0.97 | 2500 | 7.0667 |
7.0373 | 1.16 | 3000 | 6.9511 |
6.9767 | 1.36 | 3500 | 6.8339 |
6.9483 | 1.55 | 4000 | 6.7795 |
6.9071 | 1.74 | 4500 | 6.7828 |
6.8591 | 1.94 | 5000 | 6.7164 |
6.8595 | 2.13 | 5500 | 6.7705 |
6.8406 | 2.32 | 6000 | 6.6906 |
6.7861 | 2.52 | 6500 | 6.6878 |
6.8103 | 2.71 | 7000 | 6.6486 |
6.7724 | 2.9 | 7500 | 6.6703 |
6.7563 | 3.1 | 8000 | 6.6626 |
6.7567 | 3.29 | 8500 | 6.6603 |
6.7315 | 3.49 | 9000 | 6.6392 |
6.7443 | 3.68 | 9500 | 6.6306 |
6.7244 | 3.87 | 10000 | 6.6456 |
6.7464 | 4.07 | 10500 | 6.6224 |
6.7008 | 4.26 | 11000 | 6.6138 |
6.7076 | 4.45 | 11500 | 6.6783 |
6.6944 | 4.65 | 12000 | 6.6147 |
6.6993 | 4.84 | 12500 | 6.6466 |
6.6893 | 5.03 | 13000 | 6.6369 |
6.6905 | 5.23 | 13500 | 6.6293 |
6.6899 | 5.42 | 14000 | 6.6271 |
6.6835 | 5.62 | 14500 | 6.6566 |
6.6746 | 5.81 | 15000 | 6.6385 |
6.68 | 6.0 | 15500 | 6.6309 |
6.6776 | 6.2 | 16000 | 6.6069 |
6.6714 | 6.39 | 16500 | 6.5991 |
6.6766 | 6.58 | 17000 | 6.6180 |
6.6591 | 6.78 | 17500 | 6.6212 |
6.6396 | 6.97 | 18000 | 6.5804 |
6.6575 | 7.16 | 18500 | 6.6096 |
6.6506 | 7.36 | 19000 | 6.5579 |
6.6618 | 7.55 | 19500 | 6.5911 |
6.6581 | 7.75 | 20000 | 6.5870 |
6.6703 | 7.94 | 20500 | 6.6062 |
6.6392 | 8.13 | 21000 | 6.5962 |
6.6343 | 8.33 | 21500 | 6.5903 |
6.6426 | 8.52 | 22000 | 6.6010 |
6.6227 | 8.71 | 22500 | 6.6060 |
6.6392 | 8.91 | 23000 | 6.5935 |
6.6198 | 9.1 | 23500 | 6.6293 |
6.6372 | 9.3 | 24000 | 6.5594 |
6.6146 | 9.49 | 24500 | 6.5917 |
6.6119 | 9.68 | 25000 | 6.5694 |
6.6292 | 9.88 | 25500 | 6.6230 |
6.634 | 10.07 | 26000 | 6.5857 |
6.5863 | 10.26 | 26500 | 6.5938 |
6.5957 | 10.46 | 27000 | 6.6256 |
6.5928 | 10.65 | 27500 | 6.6111 |
6.5948 | 10.84 | 28000 | 6.6031 |
6.6131 | 11.04 | 28500 | 6.5582 |
6.5946 | 11.23 | 29000 | 6.6093 |
6.6155 | 11.43 | 29500 | 6.5670 |
6.6051 | 11.62 | 30000 | 6.6016 |
6.5917 | 11.81 | 30500 | 6.6045 |
6.5918 | 12.01 | 31000 | 6.5802 |
6.558 | 12.2 | 31500 | 6.5195 |
6.5896 | 12.39 | 32000 | 6.6315 |
6.5662 | 12.59 | 32500 | 6.6112 |
6.5702 | 12.78 | 33000 | 6.5779 |
6.5798 | 12.97 | 33500 | 6.5662 |
6.5963 | 13.17 | 34000 | 6.5776 |
6.5733 | 13.36 | 34500 | 6.5870 |
6.5499 | 13.56 | 35000 | 6.5850 |
6.5492 | 13.75 | 35500 | 6.5957 |
6.5466 | 13.94 | 36000 | 6.5812 |
6.5741 | 14.14 | 36500 | 6.5287 |
6.5612 | 14.33 | 37000 | 6.5611 |
6.5648 | 14.52 | 37500 | 6.5381 |
6.5661 | 14.72 | 38000 | 6.5742 |
6.5564 | 14.91 | 38500 | 6.5424 |
6.5423 | 15.1 | 39000 | 6.5987 |
6.5471 | 15.3 | 39500 | 6.5662 |
6.5559 | 15.49 | 40000 | 6.5290 |
6.5332 | 15.69 | 40500 | 6.5412 |
6.5362 | 15.88 | 41000 | 6.5486 |
6.5351 | 16.07 | 41500 | 6.5959 |
6.5337 | 16.27 | 42000 | 6.5405 |
6.5246 | 16.46 | 42500 | 6.5217 |
6.4999 | 16.65 | 43000 | 6.5443 |
6.5459 | 16.85 | 43500 | 6.5424 |
6.5077 | 17.04 | 44000 | 6.5499 |
6.5069 | 17.23 | 44500 | 6.5509 |
6.5189 | 17.43 | 45000 | 6.5310 |
6.5086 | 17.62 | 45500 | 6.5361 |
6.5182 | 17.82 | 46000 | 6.5320 |
6.51 | 18.01 | 46500 | 6.4850 |
6.4868 | 18.2 | 47000 | 6.5155 |
6.4665 | 18.4 | 47500 | 6.5305 |
6.5123 | 18.59 | 48000 | 6.5301 |
6.4981 | 18.78 | 48500 | 6.4617 |
6.4606 | 18.98 | 49000 | 6.4895 |
6.4716 | 19.17 | 49500 | 6.4790 |
6.4733 | 19.36 | 50000 | 6.4818 |
6.4935 | 19.56 | 50500 | 6.4518 |
6.4761 | 19.75 | 51000 | 6.4852 |
6.4651 | 19.95 | 51500 | 6.4836 |
6.4462 | 20.14 | 52000 | 6.4792 |
6.4605 | 20.33 | 52500 | 6.4661 |
6.4718 | 20.53 | 53000 | 6.4639 |
6.459 | 20.72 | 53500 | 6.4683 |
6.4407 | 20.91 | 54000 | 6.4663 |
6.4388 | 21.11 | 54500 | 6.4832 |
6.4479 | 21.3 | 55000 | 6.4606 |
6.4583 | 21.49 | 55500 | 6.4723 |
6.4169 | 21.69 | 56000 | 6.4897 |
6.4437 | 21.88 | 56500 | 6.4368 |
6.4566 | 22.08 | 57000 | 6.4491 |
6.4248 | 22.27 | 57500 | 6.4630 |
6.431 | 22.46 | 58000 | 6.4246 |
6.4274 | 22.66 | 58500 | 6.4618 |
6.4262 | 22.85 | 59000 | 6.4177 |
6.4328 | 23.04 | 59500 | 6.4243 |
6.4305 | 23.24 | 60000 | 6.4178 |
6.4078 | 23.43 | 60500 | 6.4310 |
6.4431 | 23.63 | 61000 | 6.4338 |
6.4066 | 23.82 | 61500 | 6.4080 |
6.417 | 24.01 | 62000 | 6.4236 |
6.4008 | 24.21 | 62500 | 6.3703 |
6.4222 | 24.4 | 63000 | 6.4188 |
6.4304 | 24.59 | 63500 | 6.3924 |
6.4063 | 24.79 | 64000 | 6.4140 |
6.4176 | 24.98 | 64500 | 6.4419 |
6.4203 | 25.17 | 65000 | 6.4250 |
6.3983 | 25.37 | 65500 | 6.3602 |
6.3911 | 25.56 | 66000 | 6.4129 |
6.3821 | 25.76 | 66500 | 6.4225 |
6.3864 | 25.95 | 67000 | 6.3801 |
6.4109 | 26.14 | 67500 | 6.4032 |
6.4136 | 26.34 | 68000 | 6.3870 |
6.3714 | 26.53 | 68500 | 6.4385 |
6.3711 | 26.72 | 69000 | 6.4081 |
6.391 | 26.92 | 69500 | 6.3901 |
6.3931 | 27.11 | 70000 | 6.4047 |
6.3842 | 27.3 | 70500 | 6.3830 |
6.3798 | 27.5 | 71000 | 6.3935 |
6.3903 | 27.69 | 71500 | 6.3756 |
6.3771 | 27.89 | 72000 | 6.3554 |
6.3763 | 28.08 | 72500 | 6.3911 |
6.3576 | 28.27 | 73000 | 6.4059 |
6.3581 | 28.47 | 73500 | 6.3976 |
6.3739 | 28.66 | 74000 | 6.3921 |
6.363 | 28.85 | 74500 | 6.3590 |
6.3687 | 29.05 | 75000 | 6.3683 |
6.3788 | 29.24 | 75500 | 6.3915 |
6.3505 | 29.43 | 76000 | 6.3826 |
6.3618 | 29.63 | 76500 | 6.3833 |
6.3287 | 29.82 | 77000 | 6.4055 |
6.3589 | 30.02 | 77500 | 6.3994 |
6.3614 | 30.21 | 78000 | 6.3848 |
6.3729 | 30.4 | 78500 | 6.3550 |
6.3687 | 30.6 | 79000 | 6.3683 |
6.3377 | 30.79 | 79500 | 6.3743 |
6.3188 | 30.98 | 80000 | 6.3113 |
6.3613 | 31.18 | 80500 | 6.3852 |
6.3428 | 31.37 | 81000 | 6.3610 |
6.3541 | 31.56 | 81500 | 6.3848 |
6.3821 | 31.76 | 82000 | 6.3706 |
6.3357 | 31.95 | 82500 | 6.3191 |
6.3408 | 32.15 | 83000 | 6.3357 |
6.3301 | 32.34 | 83500 | 6.3374 |
6.3681 | 32.53 | 84000 | 6.3583 |
6.324 | 32.73 | 84500 | 6.3472 |
6.3615 | 32.92 | 85000 | 6.3359 |
6.3382 | 33.11 | 85500 | 6.3664 |
6.34 | 33.31 | 86000 | 6.3281 |
6.3504 | 33.5 | 86500 | 6.3688 |
6.3393 | 33.69 | 87000 | 6.3553 |
6.3453 | 33.89 | 87500 | 6.3493 |
6.3293 | 34.08 | 88000 | 6.3315 |
6.3346 | 34.28 | 88500 | 6.3134 |
6.3325 | 34.47 | 89000 | 6.3631 |
6.3497 | 34.66 | 89500 | 6.3380 |
6.332 | 34.86 | 90000 | 6.3484 |
6.3224 | 35.05 | 90500 | 6.3602 |
6.3242 | 35.24 | 91000 | 6.3414 |
6.3346 | 35.44 | 91500 | 6.3151 |
6.3547 | 35.63 | 92000 | 6.3499 |
6.3243 | 35.82 | 92500 | 6.3173 |
6.3148 | 36.02 | 93000 | 6.3141 |
6.3202 | 36.21 | 93500 | 6.3358 |
6.3251 | 36.41 | 94000 | 6.2946 |
6.3313 | 36.6 | 94500 | 6.3413 |
6.3077 | 36.79 | 95000 | 6.2959 |
6.3173 | 36.99 | 95500 | 6.3220 |
6.3207 | 37.18 | 96000 | 6.3630 |
6.311 | 37.37 | 96500 | 6.3802 |
6.3259 | 37.57 | 97000 | 6.3425 |
6.3269 | 37.76 | 97500 | 6.3407 |
6.3136 | 37.96 | 98000 | 6.3140 |
6.3007 | 38.15 | 98500 | 6.3392 |
6.2911 | 38.34 | 99000 | 6.3874 |
6.3241 | 38.54 | 99500 | 6.3363 |
6.3056 | 38.73 | 100000 | 6.3766 |
6.3138 | 38.92 | 100500 | 6.3147 |
6.3065 | 39.12 | 101000 | 6.3622 |
6.3118 | 39.31 | 101500 | 6.3200 |
6.3009 | 39.5 | 102000 | 6.3316 |
6.3107 | 39.7 | 102500 | 6.3112 |
6.2977 | 39.89 | 103000 | 6.3120 |
Framework versions
- Transformers 4.24.0
- Pytorch 1.12.1+cu113
- Datasets 2.6.1
- Tokenizers 0.13.2