<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
longformer-base-4096-bne-es-finetuned-augmented1
This model is a fine-tuned version of PlanTL-GOB-ES/longformer-base-4096-bne-es on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.7936
- Precision: 0.5307
- Recall: 0.6189
- F1: 0.5714
- Accuracy: 0.8447
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
Training results
Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
---|---|---|---|---|---|---|---|
0.723 | 1.0 | 713 | 0.5777 | 0.3428 | 0.5358 | 0.4181 | 0.8332 |
0.3414 | 2.0 | 1426 | 0.7079 | 0.3337 | 0.5462 | 0.4143 | 0.8210 |
0.1307 | 3.0 | 2139 | 0.7862 | 0.4226 | 0.5868 | 0.4913 | 0.8298 |
0.0792 | 4.0 | 2852 | 0.9581 | 0.4215 | 0.5906 | 0.4919 | 0.8245 |
0.0427 | 5.0 | 3565 | 1.0090 | 0.4451 | 0.6047 | 0.5128 | 0.8303 |
0.032 | 6.0 | 4278 | 1.0855 | 0.4865 | 0.6123 | 0.5422 | 0.8450 |
0.0237 | 7.0 | 4991 | 1.1150 | 0.4693 | 0.6066 | 0.5292 | 0.8455 |
0.0171 | 8.0 | 5704 | 1.1544 | 0.4778 | 0.5991 | 0.5316 | 0.8456 |
0.0155 | 9.0 | 6417 | 1.1691 | 0.4812 | 0.6038 | 0.5356 | 0.8421 |
0.0114 | 10.0 | 7130 | 1.2833 | 0.4861 | 0.6104 | 0.5412 | 0.8349 |
0.0137 | 11.0 | 7843 | 1.2716 | 0.4594 | 0.6028 | 0.5214 | 0.8334 |
0.0104 | 12.0 | 8556 | 1.2635 | 0.4571 | 0.5981 | 0.5182 | 0.8459 |
0.0053 | 13.0 | 9269 | 1.2427 | 0.4447 | 0.6066 | 0.5132 | 0.8419 |
0.0067 | 14.0 | 9982 | 1.2834 | 0.4862 | 0.6 | 0.5372 | 0.8432 |
0.0068 | 15.0 | 10695 | 1.3774 | 0.5012 | 0.6094 | 0.5500 | 0.8373 |
0.0077 | 16.0 | 11408 | 1.3625 | 0.4871 | 0.6057 | 0.5399 | 0.8428 |
0.0051 | 17.0 | 12121 | 1.3764 | 0.5 | 0.6113 | 0.5501 | 0.8445 |
0.0061 | 18.0 | 12834 | 1.5528 | 0.4613 | 0.6009 | 0.5219 | 0.8267 |
0.0049 | 19.0 | 13547 | 1.3307 | 0.5070 | 0.6151 | 0.5558 | 0.8538 |
0.0059 | 20.0 | 14260 | 1.3556 | 0.4903 | 0.6198 | 0.5475 | 0.8439 |
0.0064 | 21.0 | 14973 | 1.4599 | 0.5004 | 0.6123 | 0.5507 | 0.8409 |
0.0057 | 22.0 | 15686 | 1.3506 | 0.5077 | 0.6217 | 0.5589 | 0.8439 |
0.0054 | 23.0 | 16399 | 1.5439 | 0.4914 | 0.5953 | 0.5384 | 0.8377 |
0.0034 | 24.0 | 17112 | 1.5174 | 0.5059 | 0.6066 | 0.5517 | 0.8377 |
0.0048 | 25.0 | 17825 | 1.5228 | 0.4984 | 0.6057 | 0.5468 | 0.8438 |
0.0041 | 26.0 | 18538 | 1.4479 | 0.5224 | 0.6057 | 0.5609 | 0.8403 |
0.0049 | 27.0 | 19251 | 1.3992 | 0.5291 | 0.6349 | 0.5772 | 0.8447 |
0.0048 | 28.0 | 19964 | 1.4971 | 0.5234 | 0.6321 | 0.5726 | 0.8478 |
0.0018 | 29.0 | 20677 | 1.4874 | 0.4981 | 0.6151 | 0.5504 | 0.8390 |
0.0035 | 30.0 | 21390 | 1.3051 | 0.5051 | 0.6094 | 0.5524 | 0.8421 |
0.0031 | 31.0 | 22103 | 1.5998 | 0.5133 | 0.6179 | 0.5608 | 0.8364 |
0.0031 | 32.0 | 22816 | 1.4274 | 0.5085 | 0.6179 | 0.5579 | 0.8458 |
0.0042 | 33.0 | 23529 | 1.3180 | 0.5111 | 0.6066 | 0.5548 | 0.8494 |
0.0022 | 34.0 | 24242 | 1.5043 | 0.4886 | 0.6085 | 0.5420 | 0.8442 |
0.0021 | 35.0 | 24955 | 1.5247 | 0.4962 | 0.6094 | 0.5470 | 0.8425 |
0.0024 | 36.0 | 25668 | 1.5139 | 0.4851 | 0.5981 | 0.5357 | 0.8432 |
0.0027 | 37.0 | 26381 | 1.5214 | 0.4930 | 0.6009 | 0.5417 | 0.8404 |
0.0024 | 38.0 | 27094 | 1.4470 | 0.5087 | 0.6075 | 0.5537 | 0.8472 |
0.0009 | 39.0 | 27807 | 1.4867 | 0.5016 | 0.6094 | 0.5503 | 0.8485 |
0.0015 | 40.0 | 28520 | 1.5234 | 0.5148 | 0.6217 | 0.5632 | 0.8483 |
0.0023 | 41.0 | 29233 | 1.5742 | 0.4926 | 0.6264 | 0.5515 | 0.8407 |
0.0017 | 42.0 | 29946 | 1.5897 | 0.5252 | 0.6 | 0.5601 | 0.8403 |
0.0022 | 43.0 | 30659 | 1.4243 | 0.4889 | 0.6038 | 0.5403 | 0.8448 |
0.001 | 44.0 | 31372 | 1.6117 | 0.5081 | 0.6179 | 0.5577 | 0.8462 |
0.0015 | 45.0 | 32085 | 1.5342 | 0.5169 | 0.6066 | 0.5582 | 0.8405 |
0.0005 | 46.0 | 32798 | 1.5110 | 0.4687 | 0.6142 | 0.5316 | 0.8432 |
0.0019 | 47.0 | 33511 | 1.5835 | 0.5066 | 0.6132 | 0.5548 | 0.8427 |
0.0063 | 48.0 | 34224 | 1.5688 | 0.5058 | 0.5802 | 0.5404 | 0.8394 |
0.0017 | 49.0 | 34937 | 1.5410 | 0.5075 | 0.6028 | 0.5511 | 0.8419 |
0.0012 | 50.0 | 35650 | 1.5343 | 0.5220 | 0.5943 | 0.5558 | 0.8359 |
0.0009 | 51.0 | 36363 | 1.5190 | 0.5173 | 0.6358 | 0.5705 | 0.8411 |
0.0006 | 52.0 | 37076 | 1.6576 | 0.5066 | 0.6189 | 0.5571 | 0.8311 |
0.0009 | 53.0 | 37789 | 1.5675 | 0.5155 | 0.6283 | 0.5663 | 0.8475 |
0.0007 | 54.0 | 38502 | 1.6993 | 0.5218 | 0.6208 | 0.5670 | 0.8328 |
0.0019 | 55.0 | 39215 | 1.6003 | 0.5284 | 0.6047 | 0.5640 | 0.8365 |
0.0014 | 56.0 | 39928 | 1.4922 | 0.5428 | 0.6226 | 0.5800 | 0.8556 |
0.0004 | 57.0 | 40641 | 1.5974 | 0.5402 | 0.6142 | 0.5748 | 0.8464 |
0.0002 | 58.0 | 41354 | 1.7351 | 0.5501 | 0.6113 | 0.5791 | 0.8417 |
0.0008 | 59.0 | 42067 | 1.6191 | 0.5179 | 0.6132 | 0.5616 | 0.8476 |
0.0006 | 60.0 | 42780 | 1.5721 | 0.5059 | 0.6094 | 0.5528 | 0.8455 |
0.0009 | 61.0 | 43493 | 1.6079 | 0.4980 | 0.6 | 0.5443 | 0.8388 |
0.0011 | 62.0 | 44206 | 1.7208 | 0.4907 | 0.5943 | 0.5375 | 0.8288 |
0.0002 | 63.0 | 44919 | 1.7335 | 0.5012 | 0.5925 | 0.5430 | 0.8354 |
0.001 | 64.0 | 45632 | 1.7670 | 0.5439 | 0.6189 | 0.5790 | 0.8352 |
0.0002 | 65.0 | 46345 | 1.7687 | 0.5203 | 0.6170 | 0.5645 | 0.8430 |
0.0002 | 66.0 | 47058 | 1.7857 | 0.5059 | 0.6066 | 0.5517 | 0.8375 |
0.0003 | 67.0 | 47771 | 1.7961 | 0.5090 | 0.6104 | 0.5551 | 0.8335 |
0.0009 | 68.0 | 48484 | 1.7072 | 0.5039 | 0.6132 | 0.5532 | 0.8416 |
0.0003 | 69.0 | 49197 | 1.7345 | 0.5147 | 0.6113 | 0.5589 | 0.8421 |
0.0002 | 70.0 | 49910 | 1.6423 | 0.5427 | 0.6179 | 0.5779 | 0.8491 |
0.0007 | 71.0 | 50623 | 1.6966 | 0.5422 | 0.6368 | 0.5857 | 0.8425 |
0.0016 | 72.0 | 51336 | 1.7376 | 0.5153 | 0.6198 | 0.5627 | 0.8349 |
0.001 | 73.0 | 52049 | 1.6447 | 0.51 | 0.6255 | 0.5619 | 0.8442 |
0.0001 | 74.0 | 52762 | 1.7449 | 0.5204 | 0.6132 | 0.5630 | 0.8421 |
0.0002 | 75.0 | 53475 | 1.6948 | 0.5287 | 0.6179 | 0.5698 | 0.8450 |
0.0005 | 76.0 | 54188 | 1.6546 | 0.5305 | 0.6321 | 0.5768 | 0.8480 |
0.0002 | 77.0 | 54901 | 1.7188 | 0.5224 | 0.6264 | 0.5697 | 0.8444 |
0.0001 | 78.0 | 55614 | 1.6167 | 0.5102 | 0.6142 | 0.5574 | 0.8462 |
0.0005 | 79.0 | 56327 | 1.6709 | 0.5160 | 0.6245 | 0.5651 | 0.8462 |
0.0 | 80.0 | 57040 | 1.6883 | 0.5223 | 0.6179 | 0.5661 | 0.8475 |
0.0002 | 81.0 | 57753 | 1.7612 | 0.5051 | 0.6057 | 0.5508 | 0.8436 |
0.0001 | 82.0 | 58466 | 1.7766 | 0.5342 | 0.6189 | 0.5734 | 0.8410 |
0.0001 | 83.0 | 59179 | 1.7235 | 0.5252 | 0.6189 | 0.5682 | 0.8453 |
0.0002 | 84.0 | 59892 | 1.7663 | 0.5319 | 0.6208 | 0.5729 | 0.8440 |
0.0007 | 85.0 | 60605 | 1.7581 | 0.5280 | 0.6217 | 0.5711 | 0.8408 |
0.0002 | 86.0 | 61318 | 1.7467 | 0.5271 | 0.6236 | 0.5713 | 0.8407 |
0.0003 | 87.0 | 62031 | 1.7220 | 0.5275 | 0.6151 | 0.5679 | 0.8437 |
0.0001 | 88.0 | 62744 | 1.7616 | 0.5207 | 0.6179 | 0.5651 | 0.8430 |
0.0 | 89.0 | 63457 | 1.7817 | 0.5396 | 0.6170 | 0.5757 | 0.8460 |
0.0 | 90.0 | 64170 | 1.7684 | 0.5319 | 0.6132 | 0.5697 | 0.8436 |
0.0 | 91.0 | 64883 | 1.7731 | 0.5264 | 0.6208 | 0.5697 | 0.8434 |
0.0 | 92.0 | 65596 | 1.7448 | 0.5314 | 0.6236 | 0.5738 | 0.8467 |
0.0 | 93.0 | 66309 | 1.7457 | 0.5353 | 0.6302 | 0.5789 | 0.8484 |
0.0 | 94.0 | 67022 | 1.7477 | 0.5424 | 0.6274 | 0.5818 | 0.8485 |
0.0 | 95.0 | 67735 | 1.7931 | 0.5292 | 0.6160 | 0.5693 | 0.8444 |
0.0002 | 96.0 | 68448 | 1.8056 | 0.5287 | 0.6170 | 0.5694 | 0.8455 |
0.0001 | 97.0 | 69161 | 1.7963 | 0.5247 | 0.6217 | 0.5691 | 0.8450 |
0.0001 | 98.0 | 69874 | 1.7963 | 0.5211 | 0.6179 | 0.5654 | 0.8446 |
0.0001 | 99.0 | 70587 | 1.7950 | 0.5261 | 0.6189 | 0.5687 | 0.8452 |
0.0002 | 100.0 | 71300 | 1.7936 | 0.5307 | 0.6189 | 0.5714 | 0.8447 |
Framework versions
- Transformers 4.28.1
- Pytorch 2.0.0+cu117
- Datasets 2.12.0
- Tokenizers 0.13.3