<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
spanish_mlm-test
This model is a fine-tuned version of che111/spanish_mlm on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.0229
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 256
- eval_batch_size: 256
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 300
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 1.0 | 29 | 6.2056 |
| No log | 2.0 | 58 | 5.2656 |
| No log | 3.0 | 87 | 4.3991 |
| 5.6534 | 4.0 | 116 | 3.8990 |
| 5.6534 | 5.0 | 145 | 3.5027 |
| 5.6534 | 6.0 | 174 | 3.1796 |
| 5.6534 | 7.0 | 203 | 2.9333 |
| 3.4402 | 8.0 | 232 | 2.7876 |
| 3.4402 | 9.0 | 261 | 2.6124 |
| 3.4402 | 10.0 | 290 | 2.5648 |
| 3.4402 | 11.0 | 319 | 2.4046 |
| 2.7185 | 12.0 | 348 | 2.3318 |
| 2.7185 | 13.0 | 377 | 2.2789 |
| 2.7185 | 14.0 | 406 | 2.2100 |
| 2.7185 | 15.0 | 435 | 2.1735 |
| 2.3867 | 16.0 | 464 | 2.1337 |
| 2.3867 | 17.0 | 493 | 2.0764 |
| 2.3867 | 18.0 | 522 | 1.9848 |
| 2.3867 | 19.0 | 551 | 1.9665 |
| 2.1706 | 20.0 | 580 | 1.9613 |
| 2.1706 | 21.0 | 609 | 1.8838 |
| 2.1706 | 22.0 | 638 | 1.8762 |
| 2.1706 | 23.0 | 667 | 1.8094 |
| 2.0165 | 24.0 | 696 | 1.7878 |
| 2.0165 | 25.0 | 725 | 1.8003 |
| 2.0165 | 26.0 | 754 | 1.7663 |
| 2.0165 | 27.0 | 783 | 1.7745 |
| 1.9005 | 28.0 | 812 | 1.7461 |
| 1.9005 | 29.0 | 841 | 1.6817 |
| 1.9005 | 30.0 | 870 | 1.6689 |
| 1.8118 | 31.0 | 899 | 1.6837 |
| 1.8118 | 32.0 | 928 | 1.6755 |
| 1.8118 | 33.0 | 957 | 1.6501 |
| 1.8118 | 34.0 | 986 | 1.6231 |
| 1.7355 | 35.0 | 1015 | 1.6412 |
| 1.7355 | 36.0 | 1044 | 1.6010 |
| 1.7355 | 37.0 | 1073 | 1.5781 |
| 1.7355 | 38.0 | 1102 | 1.5689 |
| 1.6732 | 39.0 | 1131 | 1.5261 |
| 1.6732 | 40.0 | 1160 | 1.5822 |
| 1.6732 | 41.0 | 1189 | 1.5301 |
| 1.6732 | 42.0 | 1218 | 1.5308 |
| 1.6195 | 43.0 | 1247 | 1.5241 |
| 1.6195 | 44.0 | 1276 | 1.5375 |
| 1.6195 | 45.0 | 1305 | 1.5058 |
| 1.6195 | 46.0 | 1334 | 1.4599 |
| 1.572 | 47.0 | 1363 | 1.4809 |
| 1.572 | 48.0 | 1392 | 1.4875 |
| 1.572 | 49.0 | 1421 | 1.4727 |
| 1.572 | 50.0 | 1450 | 1.4515 |
| 1.5337 | 51.0 | 1479 | 1.4434 |
| 1.5337 | 52.0 | 1508 | 1.4363 |
| 1.5337 | 53.0 | 1537 | 1.4378 |
| 1.5337 | 54.0 | 1566 | 1.4287 |
| 1.4995 | 55.0 | 1595 | 1.4178 |
| 1.4995 | 56.0 | 1624 | 1.4197 |
| 1.4995 | 57.0 | 1653 | 1.3979 |
| 1.4643 | 58.0 | 1682 | 1.3908 |
| 1.4643 | 59.0 | 1711 | 1.4155 |
| 1.4643 | 60.0 | 1740 | 1.3825 |
| 1.4643 | 61.0 | 1769 | 1.3927 |
| 1.4384 | 62.0 | 1798 | 1.3735 |
| 1.4384 | 63.0 | 1827 | 1.3627 |
| 1.4384 | 64.0 | 1856 | 1.3734 |
| 1.4384 | 65.0 | 1885 | 1.3464 |
| 1.4108 | 66.0 | 1914 | 1.3433 |
| 1.4108 | 67.0 | 1943 | 1.3657 |
| 1.4108 | 68.0 | 1972 | 1.3687 |
| 1.4108 | 69.0 | 2001 | 1.3432 |
| 1.386 | 70.0 | 2030 | 1.3183 |
| 1.386 | 71.0 | 2059 | 1.3282 |
| 1.386 | 72.0 | 2088 | 1.3095 |
| 1.386 | 73.0 | 2117 | 1.3338 |
| 1.3613 | 74.0 | 2146 | 1.3174 |
| 1.3613 | 75.0 | 2175 | 1.2883 |
| 1.3613 | 76.0 | 2204 | 1.3092 |
| 1.3613 | 77.0 | 2233 | 1.2946 |
| 1.3425 | 78.0 | 2262 | 1.2778 |
| 1.3425 | 79.0 | 2291 | 1.2610 |
| 1.3425 | 80.0 | 2320 | 1.2839 |
| 1.3425 | 81.0 | 2349 | 1.3171 |
| 1.3247 | 82.0 | 2378 | 1.2946 |
| 1.3247 | 83.0 | 2407 | 1.2650 |
| 1.3247 | 84.0 | 2436 | 1.2551 |
| 1.302 | 85.0 | 2465 | 1.2621 |
| 1.302 | 86.0 | 2494 | 1.2472 |
| 1.302 | 87.0 | 2523 | 1.2680 |
| 1.302 | 88.0 | 2552 | 1.2671 |
| 1.2827 | 89.0 | 2581 | 1.2698 |
| 1.2827 | 90.0 | 2610 | 1.2421 |
| 1.2827 | 91.0 | 2639 | 1.2185 |
| 1.2827 | 92.0 | 2668 | 1.2431 |
| 1.2688 | 93.0 | 2697 | 1.2255 |
| 1.2688 | 94.0 | 2726 | 1.2262 |
| 1.2688 | 95.0 | 2755 | 1.2346 |
| 1.2688 | 96.0 | 2784 | 1.2253 |
| 1.2567 | 97.0 | 2813 | 1.2149 |
| 1.2567 | 98.0 | 2842 | 1.2308 |
| 1.2567 | 99.0 | 2871 | 1.2536 |
| 1.2567 | 100.0 | 2900 | 1.2383 |
| 1.2433 | 101.0 | 2929 | 1.2042 |
| 1.2433 | 102.0 | 2958 | 1.2217 |
| 1.2433 | 103.0 | 2987 | 1.1983 |
| 1.2433 | 104.0 | 3016 | 1.2058 |
| 1.2271 | 105.0 | 3045 | 1.2128 |
| 1.2271 | 106.0 | 3074 | 1.1883 |
| 1.2271 | 107.0 | 3103 | 1.2038 |
| 1.2271 | 108.0 | 3132 | 1.2049 |
| 1.2195 | 109.0 | 3161 | 1.1882 |
| 1.2195 | 110.0 | 3190 | 1.2126 |
| 1.2195 | 111.0 | 3219 | 1.1865 |
| 1.2041 | 112.0 | 3248 | 1.2186 |
| 1.2041 | 113.0 | 3277 | 1.1994 |
| 1.2041 | 114.0 | 3306 | 1.1933 |
| 1.2041 | 115.0 | 3335 | 1.1601 |
| 1.1931 | 116.0 | 3364 | 1.1810 |
| 1.1931 | 117.0 | 3393 | 1.1814 |
| 1.1931 | 118.0 | 3422 | 1.1811 |
| 1.1931 | 119.0 | 3451 | 1.1829 |
| 1.1826 | 120.0 | 3480 | 1.1730 |
| 1.1826 | 121.0 | 3509 | 1.1866 |
| 1.1826 | 122.0 | 3538 | 1.1506 |
| 1.1826 | 123.0 | 3567 | 1.1904 |
| 1.1717 | 124.0 | 3596 | 1.1589 |
| 1.1717 | 125.0 | 3625 | 1.1902 |
| 1.1717 | 126.0 | 3654 | 1.1917 |
| 1.1717 | 127.0 | 3683 | 1.1570 |
| 1.1588 | 128.0 | 3712 | 1.1627 |
| 1.1588 | 129.0 | 3741 | 1.1668 |
| 1.1588 | 130.0 | 3770 | 1.1286 |
| 1.1588 | 131.0 | 3799 | 1.1477 |
| 1.1542 | 132.0 | 3828 | 1.1307 |
| 1.1542 | 133.0 | 3857 | 1.1309 |
| 1.1542 | 134.0 | 3886 | 1.1455 |
| 1.1542 | 135.0 | 3915 | 1.1597 |
| 1.1418 | 136.0 | 3944 | 1.1441 |
| 1.1418 | 137.0 | 3973 | 1.1477 |
| 1.1418 | 138.0 | 4002 | 1.1350 |
| 1.1418 | 139.0 | 4031 | 1.1563 |
| 1.1359 | 140.0 | 4060 | 1.1261 |
| 1.1359 | 141.0 | 4089 | 1.1419 |
| 1.1359 | 142.0 | 4118 | 1.1149 |
| 1.1268 | 143.0 | 4147 | 1.1389 |
| 1.1268 | 144.0 | 4176 | 1.1031 |
| 1.1268 | 145.0 | 4205 | 1.1433 |
| 1.1268 | 146.0 | 4234 | 1.1319 |
| 1.1171 | 147.0 | 4263 | 1.1318 |
| 1.1171 | 148.0 | 4292 | 1.1453 |
| 1.1171 | 149.0 | 4321 | 1.1314 |
| 1.1171 | 150.0 | 4350 | 1.1239 |
| 1.1134 | 151.0 | 4379 | 1.1290 |
| 1.1134 | 152.0 | 4408 | 1.1148 |
| 1.1134 | 153.0 | 4437 | 1.1272 |
| 1.1134 | 154.0 | 4466 | 1.1218 |
| 1.1033 | 155.0 | 4495 | 1.1259 |
| 1.1033 | 156.0 | 4524 | 1.1306 |
| 1.1033 | 157.0 | 4553 | 1.1340 |
| 1.1033 | 158.0 | 4582 | 1.1114 |
| 1.094 | 159.0 | 4611 | 1.1415 |
| 1.094 | 160.0 | 4640 | 1.1184 |
| 1.094 | 161.0 | 4669 | 1.1101 |
| 1.094 | 162.0 | 4698 | 1.0919 |
| 1.0892 | 163.0 | 4727 | 1.1161 |
| 1.0892 | 164.0 | 4756 | 1.1037 |
| 1.0892 | 165.0 | 4785 | 1.1010 |
| 1.0892 | 166.0 | 4814 | 1.1099 |
| 1.0864 | 167.0 | 4843 | 1.1079 |
| 1.0864 | 168.0 | 4872 | 1.0875 |
| 1.0864 | 169.0 | 4901 | 1.1085 |
| 1.0771 | 170.0 | 4930 | 1.1063 |
| 1.0771 | 171.0 | 4959 | 1.0844 |
| 1.0771 | 172.0 | 4988 | 1.1221 |
| 1.0771 | 173.0 | 5017 | 1.0886 |
| 1.0691 | 174.0 | 5046 | 1.1019 |
| 1.0691 | 175.0 | 5075 | 1.0966 |
| 1.0691 | 176.0 | 5104 | 1.0899 |
| 1.0691 | 177.0 | 5133 | 1.0889 |
| 1.0643 | 178.0 | 5162 | 1.0941 |
| 1.0643 | 179.0 | 5191 | 1.1090 |
| 1.0643 | 180.0 | 5220 | 1.0913 |
| 1.0643 | 181.0 | 5249 | 1.0715 |
| 1.0605 | 182.0 | 5278 | 1.0801 |
| 1.0605 | 183.0 | 5307 | 1.0928 |
| 1.0605 | 184.0 | 5336 | 1.1143 |
| 1.0605 | 185.0 | 5365 | 1.0716 |
| 1.0547 | 186.0 | 5394 | 1.0828 |
| 1.0547 | 187.0 | 5423 | 1.0955 |
| 1.0547 | 188.0 | 5452 | 1.0745 |
| 1.0547 | 189.0 | 5481 | 1.0741 |
| 1.0518 | 190.0 | 5510 | 1.0781 |
| 1.0518 | 191.0 | 5539 | 1.0971 |
| 1.0518 | 192.0 | 5568 | 1.0916 |
| 1.0518 | 193.0 | 5597 | 1.1121 |
| 1.0434 | 194.0 | 5626 | 1.0991 |
| 1.0434 | 195.0 | 5655 | 1.0745 |
| 1.0434 | 196.0 | 5684 | 1.0862 |
| 1.0388 | 197.0 | 5713 | 1.0837 |
| 1.0388 | 198.0 | 5742 | 1.0665 |
| 1.0388 | 199.0 | 5771 | 1.0320 |
| 1.0388 | 200.0 | 5800 | 1.0648 |
| 1.0349 | 201.0 | 5829 | 1.0792 |
| 1.0349 | 202.0 | 5858 | 1.0661 |
| 1.0349 | 203.0 | 5887 | 1.0804 |
| 1.0349 | 204.0 | 5916 | 1.0838 |
| 1.0335 | 205.0 | 5945 | 1.0839 |
| 1.0335 | 206.0 | 5974 | 1.0862 |
| 1.0335 | 207.0 | 6003 | 1.0787 |
| 1.0335 | 208.0 | 6032 | 1.0981 |
| 1.027 | 209.0 | 6061 | 1.0768 |
| 1.027 | 210.0 | 6090 | 1.0805 |
| 1.027 | 211.0 | 6119 | 1.0620 |
| 1.027 | 212.0 | 6148 | 1.0728 |
| 1.0269 | 213.0 | 6177 | 1.0643 |
| 1.0269 | 214.0 | 6206 | 1.0505 |
| 1.0269 | 215.0 | 6235 | 1.0724 |
| 1.0269 | 216.0 | 6264 | 1.0534 |
| 1.0207 | 217.0 | 6293 | 1.0782 |
| 1.0207 | 218.0 | 6322 | 1.0527 |
| 1.0207 | 219.0 | 6351 | 1.0566 |
| 1.0207 | 220.0 | 6380 | 1.0765 |
| 1.0176 | 221.0 | 6409 | 1.0635 |
| 1.0176 | 222.0 | 6438 | 1.0613 |
| 1.0176 | 223.0 | 6467 | 1.0326 |
| 1.0165 | 224.0 | 6496 | 1.0610 |
| 1.0165 | 225.0 | 6525 | 1.0707 |
| 1.0165 | 226.0 | 6554 | 1.0692 |
| 1.0165 | 227.0 | 6583 | 1.0323 |
| 1.0111 | 228.0 | 6612 | 1.0592 |
| 1.0111 | 229.0 | 6641 | 1.0567 |
| 1.0111 | 230.0 | 6670 | 1.0606 |
| 1.0111 | 231.0 | 6699 | 1.0468 |
| 1.0068 | 232.0 | 6728 | 1.0696 |
| 1.0068 | 233.0 | 6757 | 1.0476 |
| 1.0068 | 234.0 | 6786 | 1.0600 |
| 1.0068 | 235.0 | 6815 | 1.0687 |
| 1.0072 | 236.0 | 6844 | 1.0571 |
| 1.0072 | 237.0 | 6873 | 1.0570 |
| 1.0072 | 238.0 | 6902 | 1.0581 |
| 1.0072 | 239.0 | 6931 | 1.0599 |
| 1.0024 | 240.0 | 6960 | 1.0107 |
| 1.0024 | 241.0 | 6989 | 1.0555 |
| 1.0024 | 242.0 | 7018 | 1.0404 |
| 1.0024 | 243.0 | 7047 | 1.0488 |
| 1.0032 | 244.0 | 7076 | 1.0581 |
| 1.0032 | 245.0 | 7105 | 1.0641 |
| 1.0032 | 246.0 | 7134 | 1.0560 |
| 1.0032 | 247.0 | 7163 | 1.0679 |
| 1.0001 | 248.0 | 7192 | 1.0591 |
| 1.0001 | 249.0 | 7221 | 1.0377 |
| 1.0001 | 250.0 | 7250 | 1.0308 |
| 1.0001 | 251.0 | 7279 | 1.0657 |
| 0.9982 | 252.0 | 7308 | 1.0495 |
| 0.9982 | 253.0 | 7337 | 1.0465 |
| 0.9982 | 254.0 | 7366 | 1.0666 |
| 0.9972 | 255.0 | 7395 | 1.0310 |
| 0.9972 | 256.0 | 7424 | 1.0534 |
| 0.9972 | 257.0 | 7453 | 1.0475 |
| 0.9972 | 258.0 | 7482 | 1.0785 |
| 0.9966 | 259.0 | 7511 | 1.0585 |
| 0.9966 | 260.0 | 7540 | 1.0402 |
| 0.9966 | 261.0 | 7569 | 1.0742 |
| 0.9966 | 262.0 | 7598 | 1.0515 |
| 0.989 | 263.0 | 7627 | 1.0387 |
| 0.989 | 264.0 | 7656 | 1.0431 |
| 0.989 | 265.0 | 7685 | 1.0462 |
| 0.989 | 266.0 | 7714 | 1.0800 |
| 0.9895 | 267.0 | 7743 | 1.0430 |
| 0.9895 | 268.0 | 7772 | 1.0477 |
| 0.9895 | 269.0 | 7801 | 1.0392 |
| 0.9895 | 270.0 | 7830 | 1.0514 |
| 0.9879 | 271.0 | 7859 | 1.0701 |
| 0.9879 | 272.0 | 7888 | 1.0421 |
| 0.9879 | 273.0 | 7917 | 1.0607 |
| 0.9879 | 274.0 | 7946 | 1.0170 |
| 0.9861 | 275.0 | 7975 | 1.0333 |
| 0.9861 | 276.0 | 8004 | 1.0271 |
| 0.9861 | 277.0 | 8033 | 1.0473 |
| 0.9861 | 278.0 | 8062 | 1.0309 |
| 0.9845 | 279.0 | 8091 | 1.0426 |
| 0.9845 | 280.0 | 8120 | 1.0488 |
| 0.9845 | 281.0 | 8149 | 1.0348 |
| 0.9844 | 282.0 | 8178 | 1.0331 |
| 0.9844 | 283.0 | 8207 | 1.0226 |
| 0.9844 | 284.0 | 8236 | 1.0183 |
| 0.9844 | 285.0 | 8265 | 1.0398 |
| 0.9841 | 286.0 | 8294 | 1.0406 |
| 0.9841 | 287.0 | 8323 | 1.0119 |
| 0.9841 | 288.0 | 8352 | 1.0401 |
| 0.9841 | 289.0 | 8381 | 1.0484 |
| 0.9851 | 290.0 | 8410 | 1.0065 |
| 0.9851 | 291.0 | 8439 | 1.0767 |
| 0.9851 | 292.0 | 8468 | 1.0563 |
| 0.9851 | 293.0 | 8497 | 1.0265 |
| 0.9848 | 294.0 | 8526 | 1.0661 |
| 0.9848 | 295.0 | 8555 | 1.0324 |
| 0.9848 | 296.0 | 8584 | 1.0449 |
| 0.9848 | 297.0 | 8613 | 1.0195 |
| 0.9801 | 298.0 | 8642 | 1.0250 |
| 0.9801 | 299.0 | 8671 | 1.0486 |
| 0.9801 | 300.0 | 8700 | 1.0223 |
Framework versions
- Transformers 4.26.1
- Pytorch 2.1.0+cu121
- Datasets 2.14.6
- Tokenizers 0.13.3
