<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
2020-Q2-filtered_tweets_tok_prog
This model is a fine-tuned version of DouglasPontes/2020-Q1-full_tweets_tok on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 5.2151
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 1400
- training_steps: 2400000
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 0.03 | 8000 | 7.0102 |
| 7.2267 | 0.07 | 16000 | 6.9542 |
| 7.2267 | 0.1 | 24000 | 6.9403 |
| 6.9704 | 0.14 | 32000 | 6.8324 |
| 6.9704 | 0.17 | 40000 | 6.7820 |
| 6.8499 | 0.21 | 48000 | 6.7417 |
| 6.8499 | 0.24 | 56000 | 6.6973 |
| 6.729 | 0.28 | 64000 | 6.6547 |
| 6.729 | 0.31 | 72000 | 6.6154 |
| 6.6453 | 0.35 | 80000 | 6.5520 |
| 6.6453 | 0.38 | 88000 | 6.5154 |
| 6.555 | 0.42 | 96000 | 6.4724 |
| 6.555 | 0.45 | 104000 | 6.4253 |
| 6.4797 | 0.49 | 112000 | 6.4060 |
| 6.4797 | 0.52 | 120000 | 6.3705 |
| 6.4146 | 0.56 | 128000 | 6.3289 |
| 6.4146 | 0.59 | 136000 | 6.3175 |
| 6.3623 | 0.63 | 144000 | 6.2903 |
| 6.3623 | 0.66 | 152000 | 6.2669 |
| 6.3233 | 0.7 | 160000 | 6.2329 |
| 6.3233 | 0.73 | 168000 | 6.2148 |
| 6.2846 | 0.77 | 176000 | 6.2140 |
| 6.2846 | 0.8 | 184000 | 6.1774 |
| 6.2548 | 0.84 | 192000 | 6.1518 |
| 6.2548 | 0.87 | 200000 | 6.1421 |
| 6.2163 | 0.91 | 208000 | 6.1221 |
| 6.2163 | 0.94 | 216000 | 6.1063 |
| 6.1854 | 0.98 | 224000 | 6.0982 |
| 6.1854 | 1.01 | 232000 | 6.0752 |
| 6.157 | 1.05 | 240000 | 6.0771 |
| 6.157 | 1.08 | 248000 | 6.0410 |
| 6.1311 | 1.12 | 256000 | 6.0283 |
| 6.1311 | 1.15 | 264000 | 6.0268 |
| 6.1132 | 1.19 | 272000 | 6.0237 |
| 6.1132 | 1.22 | 280000 | 6.0218 |
| 6.0771 | 1.26 | 288000 | 5.9890 |
| 6.0771 | 1.29 | 296000 | 5.9574 |
| 6.0559 | 1.33 | 304000 | 5.9871 |
| 6.0559 | 1.36 | 312000 | 5.9688 |
| 6.0159 | 1.4 | 320000 | 5.9408 |
| 6.0159 | 1.43 | 328000 | 5.9212 |
| 6.0085 | 1.47 | 336000 | 5.9064 |
| 6.0085 | 1.5 | 344000 | 5.9124 |
| 5.9947 | 1.54 | 352000 | 5.9012 |
| 5.9947 | 1.57 | 360000 | 5.8873 |
| 5.9726 | 1.61 | 368000 | 5.8993 |
| 5.9726 | 1.64 | 376000 | 5.8968 |
| 5.9668 | 1.68 | 384000 | 5.8790 |
| 5.9668 | 1.71 | 392000 | 5.8659 |
| 5.958 | 1.75 | 400000 | 5.8856 |
| 5.958 | 1.78 | 408000 | 5.8503 |
| 5.9476 | 1.82 | 416000 | 5.8859 |
| 5.9476 | 1.85 | 424000 | 5.8909 |
| 5.9195 | 1.89 | 432000 | 5.8603 |
| 5.9195 | 1.92 | 440000 | 5.8370 |
| 5.9143 | 1.96 | 448000 | 5.8232 |
| 5.9143 | 1.99 | 456000 | 5.8213 |
| 5.8991 | 2.03 | 464000 | 5.8196 |
| 5.8991 | 2.06 | 472000 | 5.8079 |
| 5.8735 | 2.09 | 480000 | 5.7811 |
| 5.8735 | 2.13 | 488000 | 5.7851 |
| 5.855 | 2.16 | 496000 | 5.7738 |
| 5.855 | 2.2 | 504000 | 5.7488 |
| 5.8666 | 2.23 | 512000 | 5.7699 |
| 5.8666 | 2.27 | 520000 | 5.7531 |
| 5.8256 | 2.3 | 528000 | 5.7357 |
| 5.8256 | 2.34 | 536000 | 5.7426 |
| 5.8222 | 2.37 | 544000 | 5.7376 |
| 5.8222 | 2.41 | 552000 | 5.7224 |
| 5.8097 | 2.44 | 560000 | 5.7088 |
| 5.8097 | 2.48 | 568000 | 5.7054 |
| 5.8077 | 2.51 | 576000 | 5.6899 |
| 5.8077 | 2.55 | 584000 | 5.6957 |
| 5.7859 | 2.58 | 592000 | 5.6851 |
| 5.7859 | 2.62 | 600000 | 5.7154 |
| 5.7823 | 2.65 | 608000 | 5.7051 |
| 5.7823 | 2.69 | 616000 | 5.6641 |
| 5.7714 | 2.72 | 624000 | 5.6700 |
| 5.7714 | 2.76 | 632000 | 5.6546 |
| 5.7686 | 2.79 | 640000 | 5.6435 |
| 5.7686 | 2.83 | 648000 | 5.6450 |
| 5.7483 | 2.86 | 656000 | 5.6132 |
| 5.7483 | 2.9 | 664000 | 5.6289 |
| 5.7308 | 2.93 | 672000 | 5.6310 |
| 5.7308 | 2.97 | 680000 | 5.6176 |
| 5.7201 | 3.0 | 688000 | 5.6278 |
| 5.7201 | 3.04 | 696000 | 5.6315 |
| 5.7202 | 3.07 | 704000 | 5.6112 |
| 5.7202 | 3.11 | 712000 | 5.6397 |
| 5.6954 | 3.14 | 720000 | 5.5901 |
| 5.6954 | 3.18 | 728000 | 5.5947 |
| 5.6794 | 3.21 | 736000 | 5.6044 |
| 5.6794 | 3.25 | 744000 | 5.5823 |
| 5.676 | 3.28 | 752000 | 5.5610 |
| 5.676 | 3.32 | 760000 | 5.5880 |
| 5.6746 | 3.35 | 768000 | 5.5645 |
| 5.6746 | 3.39 | 776000 | 5.5577 |
| 5.6617 | 3.42 | 784000 | 5.5687 |
| 5.6617 | 3.46 | 792000 | 5.5711 |
| 5.6519 | 3.49 | 800000 | 5.5424 |
| 5.6519 | 3.53 | 808000 | 5.5436 |
| 5.6453 | 3.56 | 816000 | 5.5545 |
| 5.6453 | 3.6 | 824000 | 5.5590 |
| 5.634 | 3.63 | 832000 | 5.5475 |
| 5.634 | 3.67 | 840000 | 5.5399 |
| 5.6364 | 3.7 | 848000 | 5.5167 |
| 5.6364 | 3.74 | 856000 | 5.5586 |
| 5.642 | 3.77 | 864000 | 5.5230 |
| 5.642 | 3.81 | 872000 | 5.5323 |
| 5.6453 | 3.84 | 880000 | 5.5151 |
| 5.6453 | 3.88 | 888000 | 5.5105 |
| 5.6174 | 3.91 | 896000 | 5.5233 |
| 5.6174 | 3.95 | 904000 | 5.5111 |
| 5.6076 | 3.98 | 912000 | 5.5201 |
| 5.6076 | 4.02 | 920000 | 5.5210 |
| 5.6179 | 4.05 | 928000 | 5.5249 |
| 5.6179 | 4.08 | 936000 | 5.4997 |
| 5.6125 | 4.12 | 944000 | 5.4942 |
| 5.6125 | 4.15 | 952000 | 5.5023 |
| 5.5992 | 4.19 | 960000 | 5.5026 |
| 5.5992 | 4.22 | 968000 | 5.5102 |
| 5.6088 | 4.26 | 976000 | 5.4885 |
| 5.6088 | 4.29 | 984000 | 5.4878 |
| 5.5988 | 4.33 | 992000 | 5.4941 |
| 5.5988 | 4.36 | 1000000 | 5.4859 |
| 5.5807 | 4.4 | 1008000 | 5.5001 |
| 5.5807 | 4.43 | 1016000 | 5.4815 |
| 5.5729 | 4.47 | 1024000 | 5.4762 |
| 5.5729 | 4.5 | 1032000 | 5.4702 |
| 5.5735 | 4.54 | 1040000 | 5.4680 |
| 5.5735 | 4.57 | 1048000 | 5.4746 |
| 5.5697 | 4.61 | 1056000 | 5.4505 |
| 5.5697 | 4.64 | 1064000 | 5.4598 |
| 5.5519 | 4.68 | 1072000 | 5.4463 |
| 5.5519 | 4.71 | 1080000 | 5.4462 |
| 5.5609 | 4.75 | 1088000 | 5.4327 |
| 5.5609 | 4.78 | 1096000 | 5.4424 |
| 5.5297 | 4.82 | 1104000 | 5.4504 |
| 5.5297 | 4.85 | 1112000 | 5.4250 |
| 5.5337 | 4.89 | 1120000 | 5.4178 |
| 5.5337 | 4.92 | 1128000 | 5.4223 |
| 5.5188 | 4.96 | 1136000 | 5.4344 |
| 5.5188 | 4.99 | 1144000 | 5.4237 |
| 5.5252 | 5.03 | 1152000 | 5.4352 |
| 5.5252 | 5.06 | 1160000 | 5.4122 |
| 5.5079 | 5.1 | 1168000 | 5.3956 |
| 5.5079 | 5.13 | 1176000 | 5.4041 |
| 5.5087 | 5.17 | 1184000 | 5.4014 |
| 5.5087 | 5.2 | 1192000 | 5.4066 |
| 5.4815 | 5.24 | 1200000 | 5.4048 |
| 5.4815 | 5.27 | 1208000 | 5.4176 |
| 5.5038 | 5.31 | 1216000 | 5.3841 |
| 5.5038 | 5.34 | 1224000 | 5.4197 |
| 5.5111 | 5.38 | 1232000 | 5.4098 |
| 5.5111 | 5.41 | 1240000 | 5.3933 |
| 5.4898 | 5.45 | 1248000 | 5.3870 |
| 5.4898 | 5.48 | 1256000 | 5.3909 |
| 5.4883 | 5.52 | 1264000 | 5.3741 |
| 5.4883 | 5.55 | 1272000 | 5.3825 |
| 5.489 | 5.59 | 1280000 | 5.3820 |
| 5.489 | 5.62 | 1288000 | 5.3900 |
| 5.4895 | 5.66 | 1296000 | 5.3884 |
| 5.4895 | 5.69 | 1304000 | 5.3957 |
| 5.4738 | 5.73 | 1312000 | 5.3762 |
| 5.4738 | 5.76 | 1320000 | 5.3720 |
| 5.4736 | 5.8 | 1328000 | 5.3955 |
| 5.4736 | 5.83 | 1336000 | 5.3632 |
| 5.4768 | 5.87 | 1344000 | 5.3807 |
| 5.4768 | 5.9 | 1352000 | 5.3680 |
| 5.4676 | 5.94 | 1360000 | 5.3807 |
| 5.4676 | 5.97 | 1368000 | 5.3685 |
| 5.4728 | 6.01 | 1376000 | 5.3745 |
| 5.4728 | 6.04 | 1384000 | 5.3591 |
| 5.4594 | 6.08 | 1392000 | 5.3641 |
| 5.4594 | 6.11 | 1400000 | 5.3577 |
| 5.4551 | 6.14 | 1408000 | 5.3704 |
| 5.4551 | 6.18 | 1416000 | 5.3587 |
| 5.4434 | 6.21 | 1424000 | 5.3646 |
| 5.4434 | 6.25 | 1432000 | 5.3644 |
| 5.4479 | 6.28 | 1440000 | 5.3500 |
| 5.4479 | 6.32 | 1448000 | 5.3695 |
| 5.447 | 6.35 | 1456000 | 5.3418 |
| 5.447 | 6.39 | 1464000 | 5.3468 |
| 5.4295 | 6.42 | 1472000 | 5.3460 |
| 5.4295 | 6.46 | 1480000 | 5.3491 |
| 5.4461 | 6.49 | 1488000 | 5.3509 |
| 5.4461 | 6.53 | 1496000 | 5.3335 |
| 5.4491 | 6.56 | 1504000 | 5.3422 |
| 5.4491 | 6.6 | 1512000 | 5.3506 |
| 5.4518 | 6.63 | 1520000 | 5.3481 |
| 5.4518 | 6.67 | 1528000 | 5.3398 |
| 5.442 | 6.7 | 1536000 | 5.3202 |
| 5.442 | 6.74 | 1544000 | 5.3221 |
| 5.4266 | 6.77 | 1552000 | 5.3344 |
| 5.4266 | 6.81 | 1560000 | 5.3331 |
| 5.4185 | 6.84 | 1568000 | 5.3406 |
| 5.4185 | 6.88 | 1576000 | 5.3246 |
| 5.4162 | 6.91 | 1584000 | 5.3317 |
| 5.4162 | 6.95 | 1592000 | 5.3198 |
| 5.425 | 6.98 | 1600000 | 5.3128 |
| 5.425 | 7.02 | 1608000 | 5.3174 |
| 5.4018 | 7.05 | 1616000 | 5.3192 |
| 5.4018 | 7.09 | 1624000 | 5.3178 |
| 5.4084 | 7.12 | 1632000 | 5.3163 |
| 5.4084 | 7.16 | 1640000 | 5.3155 |
| 5.4211 | 7.19 | 1648000 | 5.3180 |
| 5.4211 | 7.23 | 1656000 | 5.3208 |
| 5.4087 | 7.26 | 1664000 | 5.3175 |
| 5.4087 | 7.3 | 1672000 | 5.3004 |
| 5.3983 | 7.33 | 1680000 | 5.3081 |
| 5.3983 | 7.37 | 1688000 | 5.3048 |
| 5.4004 | 7.4 | 1696000 | 5.3077 |
| 5.4004 | 7.44 | 1704000 | 5.2859 |
| 5.3888 | 7.47 | 1712000 | 5.3083 |
| 5.3888 | 7.51 | 1720000 | 5.3010 |
| 5.3834 | 7.54 | 1728000 | 5.2991 |
| 5.3834 | 7.58 | 1736000 | 5.2878 |
| 5.379 | 7.61 | 1744000 | 5.2785 |
| 5.379 | 7.65 | 1752000 | 5.2871 |
| 5.3872 | 7.68 | 1760000 | 5.3042 |
| 5.3872 | 7.72 | 1768000 | 5.2847 |
| 5.3891 | 7.75 | 1776000 | 5.3002 |
| 5.3891 | 7.79 | 1784000 | 5.2793 |
| 5.3915 | 7.82 | 1792000 | 5.2721 |
| 5.3915 | 7.86 | 1800000 | 5.2710 |
| 5.3786 | 7.89 | 1808000 | 5.2894 |
| 5.3786 | 7.93 | 1816000 | 5.2897 |
| 5.3802 | 7.96 | 1824000 | 5.2838 |
| 5.3802 | 8.0 | 1832000 | 5.2762 |
| 5.3681 | 8.03 | 1840000 | 5.2869 |
| 5.3681 | 8.07 | 1848000 | 5.2630 |
| 5.3658 | 8.1 | 1856000 | 5.2833 |
| 5.3658 | 8.13 | 1864000 | 5.2774 |
| 5.3674 | 8.17 | 1872000 | 5.2680 |
| 5.3674 | 8.2 | 1880000 | 5.2601 |
| 5.3626 | 8.24 | 1888000 | 5.2669 |
| 5.3626 | 8.27 | 1896000 | 5.2480 |
| 5.3588 | 8.31 | 1904000 | 5.2580 |
| 5.3588 | 8.34 | 1912000 | 5.2707 |
| 5.3503 | 8.38 | 1920000 | 5.2699 |
| 5.3503 | 8.41 | 1928000 | 5.2660 |
| 5.3505 | 8.45 | 1936000 | 5.2469 |
| 5.3505 | 8.48 | 1944000 | 5.2541 |
| 5.3543 | 8.52 | 1952000 | 5.2568 |
| 5.3543 | 8.55 | 1960000 | 5.2691 |
| 5.3503 | 8.59 | 1968000 | 5.2508 |
| 5.3503 | 8.62 | 1976000 | 5.2467 |
| 5.348 | 8.66 | 1984000 | 5.2731 |
| 5.348 | 8.69 | 1992000 | 5.2624 |
| 5.3519 | 8.73 | 2000000 | 5.2682 |
| 5.3519 | 8.76 | 2008000 | 5.2457 |
| 5.3303 | 8.8 | 2016000 | 5.2627 |
| 5.3303 | 8.83 | 2024000 | 5.2619 |
| 5.3418 | 8.87 | 2032000 | 5.2428 |
| 5.3418 | 8.9 | 2040000 | 5.2523 |
| 5.3525 | 8.94 | 2048000 | 5.2514 |
| 5.3525 | 8.97 | 2056000 | 5.2533 |
| 5.3332 | 9.01 | 2064000 | 5.2367 |
| 5.3332 | 9.04 | 2072000 | 5.2391 |
| 5.3352 | 9.08 | 2080000 | 5.2304 |
| 5.3352 | 9.11 | 2088000 | 5.2329 |
| 5.3434 | 9.15 | 2096000 | 5.2337 |
| 5.3434 | 9.18 | 2104000 | 5.2364 |
| 5.3205 | 9.22 | 2112000 | 5.2368 |
| 5.3205 | 9.25 | 2120000 | 5.2304 |
| 5.3216 | 9.29 | 2128000 | 5.2256 |
| 5.3216 | 9.32 | 2136000 | 5.2172 |
| 5.3247 | 9.36 | 2144000 | 5.2261 |
| 5.3247 | 9.39 | 2152000 | 5.2383 |
| 5.3249 | 9.43 | 2160000 | 5.2242 |
| 5.3249 | 9.46 | 2168000 | 5.2455 |
| 5.3054 | 9.5 | 2176000 | 5.2404 |
| 5.3054 | 9.53 | 2184000 | 5.2329 |
| 5.3182 | 9.57 | 2192000 | 5.2129 |
| 5.3182 | 9.6 | 2200000 | 5.2111 |
| 5.3119 | 9.64 | 2208000 | 5.2214 |
| 5.3119 | 9.67 | 2216000 | 5.2236 |
| 5.302 | 9.71 | 2224000 | 5.2206 |
| 5.302 | 9.74 | 2232000 | 5.2170 |
| 5.3074 | 9.78 | 2240000 | 5.2258 |
| 5.3074 | 9.81 | 2248000 | 5.2059 |
| 5.3098 | 9.85 | 2256000 | 5.2100 |
| 5.3098 | 9.88 | 2264000 | 5.2124 |
| 5.294 | 9.92 | 2272000 | 5.2088 |
| 5.294 | 9.95 | 2280000 | 5.2018 |
| 5.3123 | 9.99 | 2288000 | 5.2135 |
| 5.3123 | 10.02 | 2296000 | 5.2197 |
| 5.3061 | 10.06 | 2304000 | 5.2147 |
| 5.3061 | 10.09 | 2312000 | 5.2134 |
| 5.2906 | 10.13 | 2320000 | 5.2046 |
| 5.2906 | 10.16 | 2328000 | 5.2007 |
| 5.2974 | 10.19 | 2336000 | 5.2045 |
| 5.2974 | 10.23 | 2344000 | 5.2041 |
| 5.2964 | 10.26 | 2352000 | 5.1983 |
| 5.2964 | 10.3 | 2360000 | 5.2027 |
| 5.3104 | 10.33 | 2368000 | 5.1968 |
| 5.3104 | 10.37 | 2376000 | 5.2040 |
| 5.2933 | 10.4 | 2384000 | 5.2189 |
| 5.2933 | 10.44 | 2392000 | 5.2054 |
| 5.307 | 10.47 | 2400000 | 5.2138 |
Framework versions
- Transformers 4.30.2
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3
