<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
vit-small_rvl_cdip_100_examples_per_class_kd_CEKD_t5.0_a0.5
This model is a fine-tuned version of WinKawaks/vit-small-patch16-224 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.2623
- Accuracy: 0.65
- Brier Loss: 0.4803
- Nll: 3.2676
- F1 Micro: 0.65
- F1 Macro: 0.6575
- Ece: 0.1722
- Aurc: 0.1414
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 100
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll | F1 Micro | F1 Macro | Ece | Aurc |
|---|---|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 25 | 3.4916 | 0.1075 | 0.9342 | 15.2561 | 0.1075 | 0.1132 | 0.1627 | 0.8874 |
| No log | 2.0 | 50 | 2.6905 | 0.395 | 0.7423 | 8.7655 | 0.395 | 0.3694 | 0.1922 | 0.3538 |
| No log | 3.0 | 75 | 2.1229 | 0.505 | 0.6157 | 5.2850 | 0.505 | 0.4830 | 0.1716 | 0.2424 |
| No log | 4.0 | 100 | 1.9322 | 0.55 | 0.5842 | 4.6402 | 0.55 | 0.5501 | 0.1744 | 0.2156 |
| No log | 5.0 | 125 | 1.8231 | 0.5575 | 0.5788 | 4.2830 | 0.5575 | 0.5494 | 0.1777 | 0.2091 |
| No log | 6.0 | 150 | 1.7318 | 0.5875 | 0.5523 | 4.4127 | 0.5875 | 0.5864 | 0.1686 | 0.1950 |
| No log | 7.0 | 175 | 1.6652 | 0.615 | 0.5325 | 3.8720 | 0.615 | 0.6192 | 0.1654 | 0.1740 |
| No log | 8.0 | 200 | 1.5910 | 0.61 | 0.5233 | 3.2435 | 0.61 | 0.6097 | 0.1556 | 0.1702 |
| No log | 9.0 | 225 | 1.7751 | 0.59 | 0.5610 | 3.9627 | 0.59 | 0.5839 | 0.1932 | 0.1965 |
| No log | 10.0 | 250 | 1.5950 | 0.5975 | 0.5521 | 3.9360 | 0.5975 | 0.5922 | 0.1868 | 0.1886 |
| No log | 11.0 | 275 | 1.6105 | 0.6 | 0.5459 | 4.2017 | 0.6 | 0.5960 | 0.1788 | 0.1696 |
| No log | 12.0 | 300 | 1.5566 | 0.5975 | 0.5283 | 3.6344 | 0.5975 | 0.5957 | 0.1843 | 0.1758 |
| No log | 13.0 | 325 | 1.5395 | 0.6225 | 0.5344 | 3.3755 | 0.6225 | 0.6327 | 0.1725 | 0.1721 |
| No log | 14.0 | 350 | 1.5117 | 0.64 | 0.5193 | 3.7990 | 0.64 | 0.6366 | 0.1849 | 0.1659 |
| No log | 15.0 | 375 | 1.5274 | 0.6225 | 0.5381 | 3.5126 | 0.6225 | 0.6198 | 0.1837 | 0.1689 |
| No log | 16.0 | 400 | 1.3822 | 0.645 | 0.4848 | 3.5167 | 0.645 | 0.6501 | 0.1426 | 0.1384 |
| No log | 17.0 | 425 | 1.4390 | 0.6325 | 0.5345 | 3.8558 | 0.6325 | 0.6406 | 0.1859 | 0.1624 |
| No log | 18.0 | 450 | 1.3763 | 0.6425 | 0.4905 | 3.0232 | 0.6425 | 0.6446 | 0.1687 | 0.1388 |
| No log | 19.0 | 475 | 1.5017 | 0.5925 | 0.5558 | 3.9738 | 0.5925 | 0.5699 | 0.2064 | 0.1827 |
| 0.7312 | 20.0 | 500 | 1.4216 | 0.64 | 0.5092 | 3.5054 | 0.64 | 0.6394 | 0.1885 | 0.1583 |
| 0.7312 | 21.0 | 525 | 1.3999 | 0.6325 | 0.5166 | 3.6206 | 0.6325 | 0.6342 | 0.1865 | 0.1586 |
| 0.7312 | 22.0 | 550 | 1.3555 | 0.6575 | 0.5092 | 3.5815 | 0.6575 | 0.6570 | 0.1748 | 0.1565 |
| 0.7312 | 23.0 | 575 | 1.3915 | 0.6375 | 0.5065 | 3.2269 | 0.6375 | 0.6367 | 0.1712 | 0.1485 |
| 0.7312 | 24.0 | 600 | 1.4116 | 0.64 | 0.5130 | 3.7646 | 0.64 | 0.6412 | 0.1690 | 0.1624 |
| 0.7312 | 25.0 | 625 | 1.3663 | 0.64 | 0.5160 | 3.0397 | 0.64 | 0.6471 | 0.1736 | 0.1575 |
| 0.7312 | 26.0 | 650 | 1.3717 | 0.63 | 0.5097 | 3.7950 | 0.63 | 0.6379 | 0.1823 | 0.1570 |
| 0.7312 | 27.0 | 675 | 1.3229 | 0.6425 | 0.4933 | 3.5568 | 0.6425 | 0.6498 | 0.1564 | 0.1470 |
| 0.7312 | 28.0 | 700 | 1.3638 | 0.6275 | 0.5124 | 3.2988 | 0.6275 | 0.6266 | 0.1916 | 0.1600 |
| 0.7312 | 29.0 | 725 | 1.3353 | 0.6475 | 0.5013 | 3.4126 | 0.6475 | 0.6407 | 0.1747 | 0.1558 |
| 0.7312 | 30.0 | 750 | 1.3788 | 0.6325 | 0.5172 | 3.4229 | 0.6325 | 0.6329 | 0.1629 | 0.1650 |
| 0.7312 | 31.0 | 775 | 1.3021 | 0.6525 | 0.4840 | 3.2418 | 0.6525 | 0.6571 | 0.1788 | 0.1412 |
| 0.7312 | 32.0 | 800 | 1.3127 | 0.6525 | 0.5058 | 3.1876 | 0.6525 | 0.6579 | 0.1879 | 0.1525 |
| 0.7312 | 33.0 | 825 | 1.3181 | 0.64 | 0.5023 | 3.1837 | 0.64 | 0.6459 | 0.1751 | 0.1529 |
| 0.7312 | 34.0 | 850 | 1.3071 | 0.6425 | 0.4954 | 3.5271 | 0.6425 | 0.6480 | 0.1615 | 0.1496 |
| 0.7312 | 35.0 | 875 | 1.2808 | 0.655 | 0.4904 | 3.2539 | 0.655 | 0.6606 | 0.1725 | 0.1448 |
| 0.7312 | 36.0 | 900 | 1.2766 | 0.68 | 0.4771 | 3.3397 | 0.68 | 0.6823 | 0.1645 | 0.1408 |
| 0.7312 | 37.0 | 925 | 1.2751 | 0.665 | 0.4837 | 3.3390 | 0.665 | 0.6728 | 0.1723 | 0.1446 |
| 0.7312 | 38.0 | 950 | 1.2658 | 0.67 | 0.4791 | 3.2603 | 0.67 | 0.6760 | 0.1781 | 0.1407 |
| 0.7312 | 39.0 | 975 | 1.2678 | 0.66 | 0.4814 | 3.1865 | 0.66 | 0.6682 | 0.1585 | 0.1414 |
| 0.0683 | 40.0 | 1000 | 1.2737 | 0.66 | 0.4840 | 3.3466 | 0.66 | 0.6658 | 0.1699 | 0.1434 |
| 0.0683 | 41.0 | 1025 | 1.2581 | 0.66 | 0.4769 | 3.1757 | 0.66 | 0.6660 | 0.1752 | 0.1398 |
| 0.0683 | 42.0 | 1050 | 1.2734 | 0.655 | 0.4833 | 3.1843 | 0.655 | 0.6600 | 0.1721 | 0.1440 |
| 0.0683 | 43.0 | 1075 | 1.2628 | 0.66 | 0.4802 | 3.2578 | 0.66 | 0.6670 | 0.1789 | 0.1403 |
| 0.0683 | 44.0 | 1100 | 1.2717 | 0.66 | 0.4837 | 3.2573 | 0.66 | 0.6651 | 0.1584 | 0.1433 |
| 0.0683 | 45.0 | 1125 | 1.2637 | 0.6475 | 0.4791 | 3.3419 | 0.6475 | 0.6545 | 0.1736 | 0.1408 |
| 0.0683 | 46.0 | 1150 | 1.2625 | 0.6575 | 0.4797 | 3.3403 | 0.6575 | 0.6642 | 0.1597 | 0.1406 |
| 0.0683 | 47.0 | 1175 | 1.2642 | 0.6525 | 0.4791 | 3.3527 | 0.6525 | 0.6592 | 0.1731 | 0.1416 |
| 0.0683 | 48.0 | 1200 | 1.2652 | 0.655 | 0.4816 | 3.2664 | 0.655 | 0.6623 | 0.1717 | 0.1413 |
| 0.0683 | 49.0 | 1225 | 1.2646 | 0.65 | 0.4806 | 3.3371 | 0.65 | 0.6568 | 0.1758 | 0.1419 |
| 0.0683 | 50.0 | 1250 | 1.2677 | 0.65 | 0.4812 | 3.4189 | 0.65 | 0.6575 | 0.1582 | 0.1427 |
| 0.0683 | 51.0 | 1275 | 1.2657 | 0.65 | 0.4813 | 3.3393 | 0.65 | 0.6565 | 0.1748 | 0.1413 |
| 0.0683 | 52.0 | 1300 | 1.2648 | 0.655 | 0.4813 | 3.3447 | 0.655 | 0.6629 | 0.1627 | 0.1419 |
| 0.0683 | 53.0 | 1325 | 1.2650 | 0.65 | 0.4813 | 3.3350 | 0.65 | 0.6565 | 0.1780 | 0.1414 |
| 0.0683 | 54.0 | 1350 | 1.2593 | 0.655 | 0.4790 | 3.3427 | 0.655 | 0.6620 | 0.1543 | 0.1399 |
| 0.0683 | 55.0 | 1375 | 1.2648 | 0.6525 | 0.4810 | 3.3368 | 0.6525 | 0.6592 | 0.1723 | 0.1414 |
| 0.0683 | 56.0 | 1400 | 1.2608 | 0.6525 | 0.4802 | 3.2599 | 0.6525 | 0.6603 | 0.1738 | 0.1411 |
| 0.0683 | 57.0 | 1425 | 1.2639 | 0.6525 | 0.4799 | 3.3437 | 0.6525 | 0.6599 | 0.1767 | 0.1413 |
| 0.0683 | 58.0 | 1450 | 1.2631 | 0.65 | 0.4810 | 3.3401 | 0.65 | 0.6578 | 0.1667 | 0.1416 |
| 0.0683 | 59.0 | 1475 | 1.2636 | 0.6525 | 0.4803 | 3.3411 | 0.6525 | 0.6594 | 0.1690 | 0.1416 |
| 0.0391 | 60.0 | 1500 | 1.2618 | 0.6525 | 0.4796 | 3.2684 | 0.6525 | 0.6600 | 0.1813 | 0.1413 |
| 0.0391 | 61.0 | 1525 | 1.2636 | 0.6525 | 0.4807 | 3.2704 | 0.6525 | 0.6595 | 0.1673 | 0.1413 |
| 0.0391 | 62.0 | 1550 | 1.2615 | 0.65 | 0.4794 | 3.2662 | 0.65 | 0.6575 | 0.1741 | 0.1413 |
| 0.0391 | 63.0 | 1575 | 1.2630 | 0.65 | 0.4803 | 3.3417 | 0.65 | 0.6575 | 0.1752 | 0.1411 |
| 0.0391 | 64.0 | 1600 | 1.2618 | 0.65 | 0.4801 | 3.2663 | 0.65 | 0.6575 | 0.1770 | 0.1413 |
| 0.0391 | 65.0 | 1625 | 1.2622 | 0.65 | 0.4802 | 3.2698 | 0.65 | 0.6575 | 0.1686 | 0.1412 |
| 0.0391 | 66.0 | 1650 | 1.2622 | 0.65 | 0.4802 | 3.3400 | 0.65 | 0.6575 | 0.1922 | 0.1412 |
| 0.0391 | 67.0 | 1675 | 1.2625 | 0.65 | 0.4802 | 3.2694 | 0.65 | 0.6575 | 0.1801 | 0.1413 |
| 0.0391 | 68.0 | 1700 | 1.2626 | 0.65 | 0.4803 | 3.2683 | 0.65 | 0.6575 | 0.1656 | 0.1414 |
| 0.0391 | 69.0 | 1725 | 1.2631 | 0.65 | 0.4806 | 3.2696 | 0.65 | 0.6575 | 0.1722 | 0.1413 |
| 0.0391 | 70.0 | 1750 | 1.2622 | 0.65 | 0.4802 | 3.2688 | 0.65 | 0.6575 | 0.1812 | 0.1412 |
| 0.0391 | 71.0 | 1775 | 1.2626 | 0.65 | 0.4803 | 3.2676 | 0.65 | 0.6575 | 0.1845 | 0.1412 |
| 0.0391 | 72.0 | 1800 | 1.2621 | 0.65 | 0.4801 | 3.2683 | 0.65 | 0.6575 | 0.1805 | 0.1411 |
| 0.0391 | 73.0 | 1825 | 1.2626 | 0.65 | 0.4804 | 3.2683 | 0.65 | 0.6575 | 0.1665 | 0.1413 |
| 0.0391 | 74.0 | 1850 | 1.2624 | 0.65 | 0.4803 | 3.2686 | 0.65 | 0.6575 | 0.1773 | 0.1412 |
| 0.0391 | 75.0 | 1875 | 1.2624 | 0.65 | 0.4803 | 3.2682 | 0.65 | 0.6575 | 0.1807 | 0.1412 |
| 0.0391 | 76.0 | 1900 | 1.2627 | 0.65 | 0.4804 | 3.2680 | 0.65 | 0.6575 | 0.1732 | 0.1414 |
| 0.0391 | 77.0 | 1925 | 1.2625 | 0.65 | 0.4803 | 3.2673 | 0.65 | 0.6575 | 0.1715 | 0.1412 |
| 0.0391 | 78.0 | 1950 | 1.2623 | 0.65 | 0.4803 | 3.2681 | 0.65 | 0.6575 | 0.1840 | 0.1413 |
| 0.0391 | 79.0 | 1975 | 1.2624 | 0.65 | 0.4803 | 3.2678 | 0.65 | 0.6575 | 0.1773 | 0.1413 |
| 0.0385 | 80.0 | 2000 | 1.2625 | 0.65 | 0.4803 | 3.2686 | 0.65 | 0.6575 | 0.1802 | 0.1414 |
| 0.0385 | 81.0 | 2025 | 1.2625 | 0.65 | 0.4803 | 3.2677 | 0.65 | 0.6575 | 0.1773 | 0.1413 |
| 0.0385 | 82.0 | 2050 | 1.2625 | 0.65 | 0.4803 | 3.2684 | 0.65 | 0.6575 | 0.1802 | 0.1414 |
| 0.0385 | 83.0 | 2075 | 1.2624 | 0.65 | 0.4803 | 3.2679 | 0.65 | 0.6575 | 0.1823 | 0.1413 |
| 0.0385 | 84.0 | 2100 | 1.2623 | 0.65 | 0.4803 | 3.2681 | 0.65 | 0.6575 | 0.1772 | 0.1413 |
| 0.0385 | 85.0 | 2125 | 1.2624 | 0.65 | 0.4803 | 3.2677 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 86.0 | 2150 | 1.2625 | 0.65 | 0.4804 | 3.2680 | 0.65 | 0.6575 | 0.1751 | 0.1414 |
| 0.0385 | 87.0 | 2175 | 1.2623 | 0.65 | 0.4803 | 3.2677 | 0.65 | 0.6575 | 0.1772 | 0.1413 |
| 0.0385 | 88.0 | 2200 | 1.2624 | 0.65 | 0.4803 | 3.2676 | 0.65 | 0.6575 | 0.1723 | 0.1414 |
| 0.0385 | 89.0 | 2225 | 1.2623 | 0.65 | 0.4803 | 3.2679 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 90.0 | 2250 | 1.2622 | 0.65 | 0.4802 | 3.2677 | 0.65 | 0.6575 | 0.1722 | 0.1413 |
| 0.0385 | 91.0 | 2275 | 1.2623 | 0.65 | 0.4803 | 3.2678 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 92.0 | 2300 | 1.2624 | 0.65 | 0.4803 | 3.2677 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 93.0 | 2325 | 1.2623 | 0.65 | 0.4803 | 3.2679 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 94.0 | 2350 | 1.2623 | 0.65 | 0.4803 | 3.2677 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 95.0 | 2375 | 1.2623 | 0.65 | 0.4803 | 3.2676 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 96.0 | 2400 | 1.2623 | 0.65 | 0.4803 | 3.2677 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 97.0 | 2425 | 1.2623 | 0.65 | 0.4803 | 3.2677 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 98.0 | 2450 | 1.2623 | 0.65 | 0.4803 | 3.2677 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 99.0 | 2475 | 1.2623 | 0.65 | 0.4803 | 3.2676 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
| 0.0385 | 100.0 | 2500 | 1.2623 | 0.65 | 0.4803 | 3.2676 | 0.65 | 0.6575 | 0.1722 | 0.1414 |
Framework versions
- Transformers 4.28.0.dev0
- Pytorch 1.12.1+cu113
- Datasets 2.12.0
- Tokenizers 0.12.1
