<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
flan-t5-base-productdomain_instructions
This model is a fine-tuned version of google/flan-t5-base on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.7837
- Rouge1: 36.5991
- Rouge2: 15.4799
- Rougel: 34.4037
- Rougelsum: 35.4543
- Gen Len: 14.0723
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 42 | 2.0081 | 33.7683 | 13.3348 | 31.7464 | 32.3501 | 13.6988 |
| No log | 2.0 | 84 | 1.9122 | 35.8931 | 14.7035 | 33.5156 | 34.4285 | 14.2771 |
| No log | 3.0 | 126 | 1.8571 | 36.1469 | 14.7339 | 34.0229 | 34.8045 | 14.8554 |
| No log | 4.0 | 168 | 1.8285 | 34.6911 | 13.3403 | 32.4862 | 33.4576 | 14.8916 |
| No log | 5.0 | 210 | 1.8121 | 36.1848 | 14.3484 | 34.4051 | 35.0242 | 14.6627 |
| No log | 6.0 | 252 | 1.7985 | 35.8217 | 14.632 | 34.0805 | 34.7598 | 14.4217 |
| No log | 7.0 | 294 | 1.7926 | 36.7474 | 14.714 | 34.4996 | 35.471 | 14.4578 |
| No log | 8.0 | 336 | 1.7837 | 36.5991 | 15.4799 | 34.4037 | 35.4543 | 14.0723 |
| No log | 9.0 | 378 | 1.7868 | 38.5794 | 17.8009 | 36.3936 | 37.2854 | 14.0482 |
| No log | 10.0 | 420 | 1.7917 | 37.186 | 16.4516 | 34.5805 | 35.5496 | 14.6867 |
| No log | 11.0 | 462 | 1.7876 | 38.2387 | 16.8602 | 35.703 | 36.5374 | 14.3735 |
| 1.5613 | 12.0 | 504 | 1.7949 | 36.6609 | 17.1147 | 34.6186 | 35.1214 | 14.8554 |
| 1.5613 | 13.0 | 546 | 1.8045 | 38.9387 | 18.0734 | 36.7076 | 37.3858 | 14.8193 |
| 1.5613 | 14.0 | 588 | 1.8211 | 39.0697 | 16.5198 | 36.7938 | 37.3292 | 14.5663 |
| 1.5613 | 15.0 | 630 | 1.8214 | 38.2996 | 17.1678 | 36.4389 | 37.0512 | 14.8313 |
| 1.5613 | 16.0 | 672 | 1.8375 | 39.4345 | 18.0457 | 37.4487 | 38.0049 | 14.5422 |
| 1.5613 | 17.0 | 714 | 1.8668 | 36.9279 | 18.0742 | 34.8528 | 35.4754 | 14.8313 |
| 1.5613 | 18.0 | 756 | 1.8708 | 36.1653 | 17.1661 | 34.0035 | 34.7663 | 14.8795 |
| 1.5613 | 19.0 | 798 | 1.9029 | 36.9342 | 16.9662 | 34.7087 | 35.3687 | 14.7229 |
| 1.5613 | 20.0 | 840 | 1.9203 | 37.0405 | 16.51 | 34.708 | 35.4081 | 15.1446 |
| 1.5613 | 21.0 | 882 | 1.9241 | 40.1118 | 18.1251 | 37.7191 | 38.5263 | 14.7590 |
| 1.5613 | 22.0 | 924 | 1.9595 | 40.3279 | 17.7924 | 38.1031 | 38.6966 | 14.9036 |
| 1.5613 | 23.0 | 966 | 1.9486 | 38.5332 | 16.3386 | 36.3524 | 36.8476 | 15.1084 |
| 0.9347 | 24.0 | 1008 | 1.9651 | 39.0175 | 17.4398 | 36.7802 | 37.3691 | 14.6386 |
| 0.9347 | 25.0 | 1050 | 2.0215 | 37.3542 | 16.7397 | 35.1152 | 35.5948 | 15.1807 |
| 0.9347 | 26.0 | 1092 | 2.0136 | 36.1433 | 16.0566 | 33.5195 | 34.1703 | 15.3855 |
| 0.9347 | 27.0 | 1134 | 2.0317 | 37.365 | 17.3246 | 34.9103 | 35.5618 | 14.7229 |
| 0.9347 | 28.0 | 1176 | 2.0574 | 38.9994 | 18.9331 | 36.8122 | 37.2818 | 14.7590 |
| 0.9347 | 29.0 | 1218 | 2.0975 | 38.704 | 17.6156 | 36.4166 | 36.855 | 15.0843 |
| 0.9347 | 30.0 | 1260 | 2.1182 | 36.6657 | 17.2754 | 34.1387 | 34.5188 | 15.3735 |
| 0.9347 | 31.0 | 1302 | 2.1353 | 38.3665 | 17.6706 | 36.2971 | 36.9008 | 14.6386 |
| 0.9347 | 32.0 | 1344 | 2.1583 | 36.461 | 15.951 | 34.1126 | 34.7238 | 14.9639 |
| 0.9347 | 33.0 | 1386 | 2.1628 | 38.2005 | 17.8068 | 35.9379 | 36.4597 | 14.8554 |
| 0.9347 | 34.0 | 1428 | 2.1632 | 38.1226 | 17.8223 | 35.7166 | 36.3522 | 15.0964 |
| 0.9347 | 35.0 | 1470 | 2.1993 | 38.1793 | 16.4788 | 36.4238 | 36.8716 | 14.6145 |
| 0.6306 | 36.0 | 1512 | 2.2278 | 37.5943 | 18.0919 | 35.9979 | 36.0291 | 14.7108 |
| 0.6306 | 37.0 | 1554 | 2.2547 | 36.7207 | 16.675 | 34.9107 | 35.0798 | 14.7229 |
| 0.6306 | 38.0 | 1596 | 2.2688 | 36.9936 | 15.8314 | 35.4724 | 35.5883 | 14.4940 |
| 0.6306 | 39.0 | 1638 | 2.3119 | 37.5208 | 16.6074 | 35.8717 | 36.445 | 14.3253 |
| 0.6306 | 40.0 | 1680 | 2.3154 | 37.8128 | 16.9579 | 35.6907 | 36.1893 | 14.6867 |
| 0.6306 | 41.0 | 1722 | 2.3531 | 39.4845 | 17.5286 | 37.8577 | 38.3204 | 14.4458 |
| 0.6306 | 42.0 | 1764 | 2.3323 | 38.4761 | 17.295 | 36.6386 | 36.9557 | 14.6145 |
| 0.6306 | 43.0 | 1806 | 2.3743 | 38.7443 | 19.2581 | 37.1116 | 37.5985 | 14.5181 |
| 0.6306 | 44.0 | 1848 | 2.4311 | 40.3561 | 18.693 | 38.8656 | 39.3105 | 14.4337 |
| 0.6306 | 45.0 | 1890 | 2.3959 | 40.0522 | 19.397 | 38.6949 | 39.2113 | 14.3494 |
| 0.6306 | 46.0 | 1932 | 2.4536 | 38.2892 | 17.2512 | 36.3746 | 37.0066 | 14.3133 |
| 0.6306 | 47.0 | 1974 | 2.4263 | 40.1626 | 18.1146 | 38.2934 | 39.0442 | 14.5422 |
| 0.4555 | 48.0 | 2016 | 2.4762 | 38.6619 | 17.2921 | 36.7469 | 37.3807 | 14.3614 |
| 0.4555 | 49.0 | 2058 | 2.5072 | 38.2839 | 17.8954 | 36.532 | 36.9102 | 14.5181 |
| 0.4555 | 50.0 | 2100 | 2.5133 | 39.5629 | 18.1928 | 37.5546 | 38.2356 | 14.4578 |
| 0.4555 | 51.0 | 2142 | 2.5239 | 39.6734 | 17.4027 | 37.8029 | 38.0765 | 14.3253 |
| 0.4555 | 52.0 | 2184 | 2.5491 | 39.6165 | 18.1724 | 37.5788 | 38.5066 | 14.4578 |
| 0.4555 | 53.0 | 2226 | 2.5733 | 38.1501 | 18.2663 | 36.3533 | 37.0174 | 14.8554 |
| 0.4555 | 54.0 | 2268 | 2.5716 | 36.2353 | 16.133 | 34.1902 | 34.7408 | 14.7590 |
| 0.4555 | 55.0 | 2310 | 2.6192 | 37.8879 | 17.7186 | 35.9678 | 36.6746 | 14.9036 |
| 0.4555 | 56.0 | 2352 | 2.6474 | 37.1621 | 17.0886 | 35.4221 | 35.731 | 14.5181 |
| 0.4555 | 57.0 | 2394 | 2.6623 | 37.5523 | 16.7998 | 35.4469 | 36.0076 | 14.4699 |
| 0.4555 | 58.0 | 2436 | 2.6607 | 38.0032 | 17.0229 | 36.1551 | 36.5535 | 14.1807 |
| 0.4555 | 59.0 | 2478 | 2.7150 | 38.1025 | 17.4752 | 36.5283 | 36.7015 | 14.2289 |
| 0.3508 | 60.0 | 2520 | 2.6941 | 39.797 | 19.2379 | 38.1214 | 38.2261 | 14.3614 |
| 0.3508 | 61.0 | 2562 | 2.7107 | 38.8625 | 17.623 | 36.6963 | 37.0603 | 14.1325 |
| 0.3508 | 62.0 | 2604 | 2.6814 | 37.5211 | 16.4479 | 35.5462 | 35.8889 | 14.3494 |
| 0.3508 | 63.0 | 2646 | 2.7484 | 38.6866 | 17.6612 | 36.7428 | 37.1636 | 14.0723 |
| 0.3508 | 64.0 | 2688 | 2.7395 | 38.0483 | 17.6948 | 36.2878 | 36.697 | 14.1807 |
| 0.3508 | 65.0 | 2730 | 2.7365 | 37.6712 | 17.2705 | 35.8893 | 36.3441 | 14.4458 |
| 0.3508 | 66.0 | 2772 | 2.7555 | 37.9902 | 17.7247 | 36.0837 | 36.7237 | 14.3012 |
| 0.3508 | 67.0 | 2814 | 2.7494 | 36.6603 | 16.2134 | 34.6886 | 35.287 | 14.5783 |
| 0.3508 | 68.0 | 2856 | 2.7826 | 37.4075 | 16.5272 | 35.4471 | 35.8108 | 14.4458 |
| 0.3508 | 69.0 | 2898 | 2.7913 | 37.5132 | 16.5865 | 35.5267 | 35.8753 | 14.3133 |
| 0.3508 | 70.0 | 2940 | 2.8110 | 38.0779 | 17.5734 | 36.2356 | 36.4576 | 14.1687 |
| 0.3508 | 71.0 | 2982 | 2.8468 | 38.0068 | 17.1148 | 35.834 | 36.2888 | 14.2289 |
| 0.2859 | 72.0 | 3024 | 2.8722 | 37.0923 | 17.2183 | 35.4736 | 35.5467 | 14.2048 |
| 0.2859 | 73.0 | 3066 | 2.8532 | 37.3506 | 17.381 | 35.5293 | 35.7809 | 14.1928 |
| 0.2859 | 74.0 | 3108 | 2.8052 | 36.9958 | 16.5001 | 35.0384 | 35.4851 | 14.3735 |
| 0.2859 | 75.0 | 3150 | 2.8523 | 37.1479 | 15.9411 | 35.287 | 35.7899 | 14.3855 |
| 0.2859 | 76.0 | 3192 | 2.8778 | 36.8889 | 15.6829 | 34.905 | 35.3649 | 14.4337 |
| 0.2859 | 77.0 | 3234 | 2.9079 | 36.5824 | 15.5738 | 34.6425 | 35.1927 | 14.3614 |
| 0.2859 | 78.0 | 3276 | 2.8787 | 36.1728 | 15.938 | 34.4013 | 34.8261 | 14.4819 |
| 0.2859 | 79.0 | 3318 | 2.9080 | 35.9696 | 15.6976 | 34.2352 | 34.5983 | 14.6386 |
| 0.2859 | 80.0 | 3360 | 2.8772 | 37.0747 | 16.8528 | 35.1818 | 35.5885 | 14.4217 |
| 0.2859 | 81.0 | 3402 | 2.9020 | 36.3635 | 17.4462 | 34.3583 | 34.9417 | 14.4819 |
| 0.2859 | 82.0 | 3444 | 2.8993 | 37.4704 | 17.335 | 35.6702 | 36.1192 | 14.4217 |
| 0.2859 | 83.0 | 3486 | 2.8920 | 37.1973 | 17.3126 | 35.4618 | 35.8107 | 14.5542 |
| 0.2455 | 84.0 | 3528 | 2.9112 | 37.3907 | 17.2948 | 35.5391 | 35.9917 | 14.5783 |
| 0.2455 | 85.0 | 3570 | 2.9250 | 36.3332 | 16.2698 | 34.4579 | 34.7125 | 14.4337 |
| 0.2455 | 86.0 | 3612 | 2.9090 | 37.8226 | 17.3181 | 35.8265 | 36.4089 | 14.2048 |
| 0.2455 | 87.0 | 3654 | 2.9097 | 37.5181 | 17.2305 | 35.5447 | 35.9105 | 14.4940 |
| 0.2455 | 88.0 | 3696 | 2.9120 | 36.5995 | 16.6394 | 34.8092 | 35.1975 | 14.6867 |
| 0.2455 | 89.0 | 3738 | 2.9235 | 37.3048 | 16.939 | 35.3615 | 35.741 | 14.4578 |
| 0.2455 | 90.0 | 3780 | 2.9270 | 37.6118 | 17.4867 | 35.656 | 36.0439 | 14.6145 |
| 0.2455 | 91.0 | 3822 | 2.9260 | 37.6441 | 17.5091 | 35.7376 | 36.113 | 14.4578 |
| 0.2455 | 92.0 | 3864 | 2.9432 | 37.4994 | 17.3906 | 35.593 | 35.977 | 14.2651 |
| 0.2455 | 93.0 | 3906 | 2.9525 | 37.3703 | 17.3245 | 35.4908 | 35.9012 | 14.5663 |
| 0.2455 | 94.0 | 3948 | 2.9546 | 36.9876 | 17.1669 | 35.1814 | 35.5809 | 14.5542 |
| 0.2455 | 95.0 | 3990 | 2.9584 | 37.1337 | 17.1325 | 35.3505 | 35.6894 | 14.5542 |
| 0.2247 | 96.0 | 4032 | 2.9607 | 36.8183 | 16.9985 | 35.0273 | 35.3368 | 14.6024 |
| 0.2247 | 97.0 | 4074 | 2.9630 | 36.8418 | 17.027 | 35.0511 | 35.3509 | 14.6145 |
| 0.2247 | 98.0 | 4116 | 2.9610 | 36.8814 | 17.027 | 35.1067 | 35.4699 | 14.5663 |
| 0.2247 | 99.0 | 4158 | 2.9581 | 36.8814 | 17.027 | 35.1067 | 35.4699 | 14.5663 |
| 0.2247 | 100.0 | 4200 | 2.9576 | 36.8814 | 17.027 | 35.1067 | 35.4699 | 14.5663 |
Framework versions
- Transformers 4.29.2
- Pytorch 2.0.1+cu118
- Datasets 2.12.0
- Tokenizers 0.13.3
