<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
vit-base-txoriaktxori
This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the txoriak_txori dataset. It achieves the following results on the evaluation set:
- Loss: 0.0559
- Accuracy: 0.9864
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy |
---|---|---|---|---|
5.8505 | 0.02 | 100 | 5.8381 | 0.2584 |
5.259 | 0.04 | 200 | 5.2556 | 0.4992 |
4.6643 | 0.06 | 300 | 4.5950 | 0.6532 |
4.0801 | 0.08 | 400 | 3.9534 | 0.6976 |
3.3312 | 0.1 | 500 | 3.2908 | 0.7608 |
2.773 | 0.12 | 600 | 2.6892 | 0.7704 |
2.3108 | 0.14 | 700 | 2.0982 | 0.7976 |
1.662 | 0.16 | 800 | 1.6214 | 0.8216 |
1.3897 | 0.18 | 900 | 1.2662 | 0.8604 |
1.1634 | 0.2 | 1000 | 0.9868 | 0.8892 |
1.0498 | 0.22 | 1100 | 0.7855 | 0.8992 |
0.5978 | 0.24 | 1200 | 0.6305 | 0.912 |
0.6399 | 0.26 | 1300 | 0.5560 | 0.9164 |
0.607 | 0.28 | 1400 | 0.5119 | 0.9192 |
0.6595 | 0.3 | 1500 | 0.4307 | 0.9272 |
0.5239 | 0.32 | 1600 | 0.4124 | 0.9176 |
0.5166 | 0.34 | 1700 | 0.3280 | 0.9312 |
0.5352 | 0.36 | 1800 | 0.3155 | 0.9308 |
0.4036 | 0.38 | 1900 | 0.2893 | 0.9424 |
0.3836 | 0.4 | 2000 | 0.3161 | 0.9272 |
0.3418 | 0.42 | 2100 | 0.3005 | 0.9384 |
0.4172 | 0.44 | 2200 | 0.2518 | 0.9456 |
0.4293 | 0.46 | 2300 | 0.2367 | 0.9424 |
0.3551 | 0.48 | 2400 | 0.2422 | 0.9432 |
0.2718 | 0.5 | 2500 | 0.2207 | 0.9492 |
0.3802 | 0.52 | 2600 | 0.2163 | 0.9428 |
0.2916 | 0.54 | 2700 | 0.2156 | 0.946 |
0.3384 | 0.56 | 2800 | 0.2037 | 0.9508 |
0.352 | 0.58 | 2900 | 0.2241 | 0.9432 |
0.3868 | 0.6 | 3000 | 0.2525 | 0.9428 |
0.3195 | 0.62 | 3100 | 0.2032 | 0.9496 |
0.2618 | 0.64 | 3200 | 0.2088 | 0.944 |
0.326 | 0.66 | 3300 | 0.1744 | 0.9536 |
0.2691 | 0.68 | 3400 | 0.1853 | 0.9516 |
0.2629 | 0.7 | 3500 | 0.1788 | 0.9464 |
0.2965 | 0.72 | 3600 | 0.1719 | 0.9572 |
0.3565 | 0.74 | 3700 | 0.2041 | 0.9452 |
0.2344 | 0.76 | 3800 | 0.1863 | 0.9504 |
0.4416 | 0.78 | 3900 | 0.1938 | 0.9472 |
0.2901 | 0.8 | 4000 | 0.1674 | 0.9572 |
0.3158 | 0.82 | 4100 | 0.2006 | 0.9496 |
0.3708 | 0.84 | 4200 | 0.1850 | 0.952 |
0.2636 | 0.86 | 4300 | 0.1488 | 0.9624 |
0.1764 | 0.88 | 4400 | 0.1818 | 0.9524 |
0.4299 | 0.9 | 4500 | 0.1642 | 0.9576 |
0.4862 | 0.92 | 4600 | 0.1867 | 0.9516 |
0.288 | 0.94 | 4700 | 0.1362 | 0.9604 |
0.2715 | 0.96 | 4800 | 0.1384 | 0.9668 |
0.3139 | 0.98 | 4900 | 0.1607 | 0.956 |
0.2301 | 1.0 | 5000 | 0.1428 | 0.9628 |
0.1527 | 1.02 | 5100 | 0.1313 | 0.9672 |
0.1856 | 1.04 | 5200 | 0.1356 | 0.9628 |
0.1143 | 1.06 | 5300 | 0.1469 | 0.962 |
0.1465 | 1.08 | 5400 | 0.1320 | 0.9648 |
0.1342 | 1.1 | 5500 | 0.1291 | 0.9644 |
0.1686 | 1.12 | 5600 | 0.1589 | 0.952 |
0.0683 | 1.14 | 5700 | 0.1598 | 0.9592 |
0.095 | 1.16 | 5800 | 0.1330 | 0.9628 |
0.1458 | 1.18 | 5900 | 0.1307 | 0.9652 |
0.2321 | 1.2 | 6000 | 0.1498 | 0.9608 |
0.0593 | 1.22 | 6100 | 0.1393 | 0.9636 |
0.1721 | 1.24 | 6200 | 0.1564 | 0.9604 |
0.2735 | 1.26 | 6300 | 0.1509 | 0.9572 |
0.1384 | 1.28 | 6400 | 0.1526 | 0.958 |
0.1232 | 1.3 | 6500 | 0.1560 | 0.9596 |
0.1615 | 1.32 | 6600 | 0.1348 | 0.9652 |
0.2521 | 1.34 | 6700 | 0.1223 | 0.9684 |
0.0616 | 1.36 | 6800 | 0.1556 | 0.9616 |
0.23 | 1.38 | 6900 | 0.1338 | 0.9652 |
0.237 | 1.4 | 7000 | 0.1140 | 0.9664 |
0.2572 | 1.42 | 7100 | 0.1191 | 0.9672 |
0.1841 | 1.44 | 7200 | 0.1121 | 0.9708 |
0.1212 | 1.46 | 7300 | 0.1089 | 0.9708 |
0.1436 | 1.48 | 7400 | 0.1246 | 0.9672 |
0.1403 | 1.5 | 7500 | 0.1234 | 0.9676 |
0.1794 | 1.52 | 7600 | 0.1273 | 0.966 |
0.2153 | 1.54 | 7700 | 0.1423 | 0.964 |
0.1347 | 1.56 | 7800 | 0.0985 | 0.9708 |
0.1989 | 1.58 | 7900 | 0.1117 | 0.9712 |
0.2686 | 1.6 | 8000 | 0.1166 | 0.9704 |
0.134 | 1.62 | 8100 | 0.1391 | 0.962 |
0.2474 | 1.64 | 8200 | 0.1280 | 0.9676 |
0.0635 | 1.66 | 8300 | 0.1079 | 0.9696 |
0.1073 | 1.68 | 8400 | 0.1335 | 0.9628 |
0.1483 | 1.7 | 8500 | 0.1108 | 0.9692 |
0.0933 | 1.72 | 8600 | 0.1059 | 0.9708 |
0.1204 | 1.74 | 8700 | 0.1007 | 0.9752 |
0.1051 | 1.76 | 8800 | 0.1055 | 0.9712 |
0.1509 | 1.78 | 8900 | 0.0995 | 0.9704 |
0.1404 | 1.8 | 9000 | 0.1012 | 0.9744 |
0.0502 | 1.82 | 9100 | 0.0913 | 0.9768 |
0.3038 | 1.84 | 9200 | 0.0988 | 0.9732 |
0.1651 | 1.86 | 9300 | 0.1146 | 0.9656 |
0.1047 | 1.88 | 9400 | 0.1140 | 0.9664 |
0.1639 | 1.9 | 9500 | 0.1059 | 0.97 |
0.1044 | 1.92 | 9600 | 0.1012 | 0.9744 |
0.1955 | 1.94 | 9700 | 0.1119 | 0.9676 |
0.1903 | 1.96 | 9800 | 0.1127 | 0.9716 |
0.1328 | 1.98 | 9900 | 0.1199 | 0.9628 |
0.1219 | 2.0 | 10000 | 0.1011 | 0.972 |
0.0514 | 2.02 | 10100 | 0.1040 | 0.9728 |
0.0194 | 2.04 | 10200 | 0.0994 | 0.9752 |
0.0469 | 2.06 | 10300 | 0.1027 | 0.9716 |
0.0417 | 2.08 | 10400 | 0.1045 | 0.9748 |
0.0566 | 2.1 | 10500 | 0.0861 | 0.9792 |
0.0427 | 2.12 | 10600 | 0.1094 | 0.974 |
0.1358 | 2.14 | 10700 | 0.0795 | 0.9776 |
0.0119 | 2.16 | 10800 | 0.0972 | 0.9748 |
0.0379 | 2.18 | 10900 | 0.1087 | 0.97 |
0.0951 | 2.2 | 11000 | 0.1079 | 0.9728 |
0.0256 | 2.22 | 11100 | 0.0951 | 0.9748 |
0.076 | 2.24 | 11200 | 0.0945 | 0.9764 |
0.1004 | 2.26 | 11300 | 0.0870 | 0.9788 |
0.0657 | 2.28 | 11400 | 0.1073 | 0.974 |
0.0332 | 2.3 | 11500 | 0.0960 | 0.9752 |
0.0087 | 2.32 | 11600 | 0.0865 | 0.978 |
0.0351 | 2.34 | 11700 | 0.0963 | 0.9736 |
0.0127 | 2.36 | 11800 | 0.0989 | 0.976 |
0.0447 | 2.38 | 11900 | 0.1038 | 0.9752 |
0.023 | 2.4 | 12000 | 0.0919 | 0.9744 |
0.0329 | 2.42 | 12100 | 0.0857 | 0.9796 |
0.042 | 2.44 | 12200 | 0.0812 | 0.9804 |
0.0549 | 2.46 | 12300 | 0.1114 | 0.9732 |
0.0806 | 2.48 | 12400 | 0.0971 | 0.9772 |
0.1768 | 2.5 | 12500 | 0.0933 | 0.974 |
0.059 | 2.52 | 12600 | 0.0943 | 0.9788 |
0.0184 | 2.54 | 12700 | 0.0874 | 0.978 |
0.021 | 2.56 | 12800 | 0.0903 | 0.9764 |
0.0457 | 2.58 | 12900 | 0.0999 | 0.976 |
0.0788 | 2.6 | 13000 | 0.0954 | 0.9732 |
0.0599 | 2.62 | 13100 | 0.0876 | 0.9752 |
0.1041 | 2.64 | 13200 | 0.1017 | 0.9744 |
0.0309 | 2.66 | 13300 | 0.0918 | 0.9772 |
0.1347 | 2.68 | 13400 | 0.0758 | 0.9792 |
0.0432 | 2.7 | 13500 | 0.0790 | 0.9808 |
0.0802 | 2.72 | 13600 | 0.0860 | 0.9776 |
0.0841 | 2.74 | 13700 | 0.0857 | 0.98 |
0.0513 | 2.76 | 13800 | 0.0895 | 0.9764 |
0.0129 | 2.78 | 13900 | 0.0861 | 0.9772 |
0.1279 | 2.8 | 14000 | 0.0895 | 0.9764 |
0.0074 | 2.82 | 14100 | 0.0842 | 0.978 |
0.0132 | 2.84 | 14200 | 0.0742 | 0.9796 |
0.0974 | 2.86 | 14300 | 0.0854 | 0.9776 |
0.0803 | 2.88 | 14400 | 0.0769 | 0.9804 |
0.037 | 2.9 | 14500 | 0.0806 | 0.9788 |
0.0936 | 2.92 | 14600 | 0.0824 | 0.9812 |
0.0064 | 2.94 | 14700 | 0.0748 | 0.9832 |
0.0631 | 2.96 | 14800 | 0.0761 | 0.9828 |
0.0158 | 2.98 | 14900 | 0.0709 | 0.9848 |
0.0433 | 3.0 | 15000 | 0.0704 | 0.9828 |
0.0028 | 3.02 | 15100 | 0.0712 | 0.9824 |
0.0031 | 3.04 | 15200 | 0.0717 | 0.9808 |
0.0191 | 3.06 | 15300 | 0.0716 | 0.9828 |
0.0051 | 3.08 | 15400 | 0.0708 | 0.9832 |
0.0205 | 3.1 | 15500 | 0.0686 | 0.9828 |
0.1147 | 3.12 | 15600 | 0.0670 | 0.984 |
0.0014 | 3.14 | 15700 | 0.0628 | 0.9848 |
0.0082 | 3.16 | 15800 | 0.0659 | 0.984 |
0.0149 | 3.18 | 15900 | 0.0672 | 0.9836 |
0.0056 | 3.2 | 16000 | 0.0676 | 0.9852 |
0.0059 | 3.22 | 16100 | 0.0706 | 0.9836 |
0.0198 | 3.24 | 16200 | 0.0725 | 0.9812 |
0.0019 | 3.26 | 16300 | 0.0681 | 0.9828 |
0.0013 | 3.28 | 16400 | 0.0681 | 0.9856 |
0.0663 | 3.3 | 16500 | 0.0704 | 0.9852 |
0.0024 | 3.32 | 16600 | 0.0697 | 0.984 |
0.0081 | 3.34 | 16700 | 0.0679 | 0.9852 |
0.0264 | 3.36 | 16800 | 0.0631 | 0.9872 |
0.0061 | 3.38 | 16900 | 0.0651 | 0.9848 |
0.0169 | 3.4 | 17000 | 0.0655 | 0.9828 |
0.0013 | 3.42 | 17100 | 0.0661 | 0.9836 |
0.0072 | 3.44 | 17200 | 0.0633 | 0.9848 |
0.009 | 3.46 | 17300 | 0.0634 | 0.9848 |
0.0028 | 3.48 | 17400 | 0.0634 | 0.9844 |
0.0024 | 3.5 | 17500 | 0.0637 | 0.9836 |
0.0031 | 3.52 | 17600 | 0.0641 | 0.9848 |
0.004 | 3.54 | 17700 | 0.0619 | 0.9856 |
0.0562 | 3.56 | 17800 | 0.0673 | 0.9856 |
0.0005 | 3.58 | 17900 | 0.0644 | 0.9864 |
0.0079 | 3.6 | 18000 | 0.0647 | 0.9872 |
0.0016 | 3.62 | 18100 | 0.0617 | 0.9872 |
0.0019 | 3.64 | 18200 | 0.0636 | 0.9872 |
0.0047 | 3.66 | 18300 | 0.0608 | 0.9848 |
0.0327 | 3.68 | 18400 | 0.0586 | 0.9868 |
0.0108 | 3.7 | 18500 | 0.0594 | 0.9872 |
0.0061 | 3.72 | 18600 | 0.0597 | 0.9868 |
0.0106 | 3.74 | 18700 | 0.0579 | 0.9872 |
0.001 | 3.76 | 18800 | 0.0564 | 0.9872 |
0.012 | 3.78 | 18900 | 0.0561 | 0.9876 |
0.0038 | 3.8 | 19000 | 0.0566 | 0.9868 |
0.0099 | 3.82 | 19100 | 0.0573 | 0.9864 |
0.0026 | 3.84 | 19200 | 0.0575 | 0.9864 |
0.0062 | 3.86 | 19300 | 0.0573 | 0.9872 |
0.0239 | 3.88 | 19400 | 0.0573 | 0.9864 |
0.0026 | 3.9 | 19500 | 0.0568 | 0.9868 |
0.0014 | 3.92 | 19600 | 0.0557 | 0.9868 |
0.0019 | 3.94 | 19700 | 0.0562 | 0.9864 |
0.0484 | 3.96 | 19800 | 0.0560 | 0.9864 |
0.0022 | 3.98 | 19900 | 0.0559 | 0.9864 |
0.0145 | 4.0 | 20000 | 0.0559 | 0.9864 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.1
- Tokenizers 0.13.2