<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
rvlcdip-tiny_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5
This model is a fine-tuned version of WinKawaks/vit-tiny-patch16-224 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.6215
- Accuracy: 0.7963
- Brier Loss: 0.3076
- Nll: 1.6291
- F1 Micro: 0.7963
- F1 Macro: 0.7978
- Ece: 0.0919
- Aurc: 0.0682
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll | F1 Micro | F1 Macro | Ece | Aurc |
---|---|---|---|---|---|---|---|---|---|---|
No log | 1.0 | 125 | 1.3808 | 0.541 | 0.5996 | 3.3159 | 0.541 | 0.5235 | 0.1039 | 0.2209 |
No log | 2.0 | 250 | 1.0577 | 0.6525 | 0.4662 | 2.6310 | 0.6525 | 0.6396 | 0.0871 | 0.1302 |
No log | 3.0 | 375 | 0.9165 | 0.7075 | 0.4104 | 2.2685 | 0.7075 | 0.7041 | 0.0788 | 0.1048 |
1.3004 | 4.0 | 500 | 0.8505 | 0.7298 | 0.3804 | 2.1171 | 0.7298 | 0.7380 | 0.0622 | 0.0934 |
1.3004 | 5.0 | 625 | 0.8063 | 0.745 | 0.3603 | 2.1178 | 0.745 | 0.7359 | 0.0588 | 0.0814 |
1.3004 | 6.0 | 750 | 0.7441 | 0.7662 | 0.3348 | 1.9219 | 0.7663 | 0.7636 | 0.0545 | 0.0741 |
1.3004 | 7.0 | 875 | 0.6987 | 0.7732 | 0.3193 | 1.8601 | 0.7732 | 0.7741 | 0.0509 | 0.0697 |
0.4682 | 8.0 | 1000 | 0.7033 | 0.773 | 0.3240 | 1.8889 | 0.7730 | 0.7733 | 0.0516 | 0.0776 |
0.4682 | 9.0 | 1125 | 0.6973 | 0.7865 | 0.3151 | 1.9589 | 0.7865 | 0.7838 | 0.0441 | 0.0760 |
0.4682 | 10.0 | 1250 | 0.7068 | 0.7748 | 0.3252 | 2.0362 | 0.7748 | 0.7749 | 0.0515 | 0.0791 |
0.4682 | 11.0 | 1375 | 0.6988 | 0.7768 | 0.3285 | 1.9227 | 0.7768 | 0.7801 | 0.0555 | 0.0840 |
0.1899 | 12.0 | 1500 | 0.7048 | 0.7762 | 0.3303 | 1.9777 | 0.7762 | 0.7719 | 0.0627 | 0.0809 |
0.1899 | 13.0 | 1625 | 0.6842 | 0.7785 | 0.3240 | 1.9360 | 0.7785 | 0.7784 | 0.0614 | 0.0808 |
0.1899 | 14.0 | 1750 | 0.6993 | 0.7742 | 0.3319 | 1.9508 | 0.7742 | 0.7727 | 0.0731 | 0.0759 |
0.1899 | 15.0 | 1875 | 0.6936 | 0.7742 | 0.3333 | 1.9042 | 0.7742 | 0.7760 | 0.0717 | 0.0853 |
0.1304 | 16.0 | 2000 | 0.6818 | 0.7837 | 0.3233 | 1.9541 | 0.7837 | 0.7855 | 0.0713 | 0.0853 |
0.1304 | 17.0 | 2125 | 0.6757 | 0.78 | 0.3255 | 1.8818 | 0.78 | 0.7829 | 0.0755 | 0.0834 |
0.1304 | 18.0 | 2250 | 0.7018 | 0.781 | 0.3348 | 2.0078 | 0.7810 | 0.7829 | 0.0786 | 0.0876 |
0.1304 | 19.0 | 2375 | 0.6872 | 0.7775 | 0.3340 | 1.8345 | 0.7775 | 0.7786 | 0.0864 | 0.0787 |
0.11 | 20.0 | 2500 | 0.7054 | 0.7758 | 0.3379 | 1.9542 | 0.7758 | 0.7747 | 0.0731 | 0.0847 |
0.11 | 21.0 | 2625 | 0.7006 | 0.782 | 0.3371 | 1.8610 | 0.782 | 0.7813 | 0.0821 | 0.0891 |
0.11 | 22.0 | 2750 | 0.7046 | 0.775 | 0.3428 | 1.8464 | 0.775 | 0.7772 | 0.0833 | 0.0814 |
0.11 | 23.0 | 2875 | 0.6620 | 0.789 | 0.3201 | 1.8174 | 0.7890 | 0.7908 | 0.0761 | 0.0799 |
0.0979 | 24.0 | 3000 | 0.6886 | 0.783 | 0.3324 | 1.8706 | 0.7830 | 0.7848 | 0.0807 | 0.0773 |
0.0979 | 25.0 | 3125 | 0.6600 | 0.7847 | 0.3236 | 1.8218 | 0.7847 | 0.7863 | 0.0833 | 0.0749 |
0.0979 | 26.0 | 3250 | 0.6777 | 0.7798 | 0.3349 | 1.7189 | 0.7798 | 0.7812 | 0.0951 | 0.0752 |
0.0979 | 27.0 | 3375 | 0.6554 | 0.7857 | 0.3212 | 1.7356 | 0.7857 | 0.7888 | 0.0871 | 0.0709 |
0.087 | 28.0 | 3500 | 0.6460 | 0.7955 | 0.3140 | 1.7680 | 0.7955 | 0.7970 | 0.0761 | 0.0696 |
0.087 | 29.0 | 3625 | 0.6371 | 0.7935 | 0.3136 | 1.6350 | 0.7935 | 0.7946 | 0.0830 | 0.0706 |
0.087 | 30.0 | 3750 | 0.6334 | 0.7915 | 0.3127 | 1.7187 | 0.7915 | 0.7933 | 0.0857 | 0.0712 |
0.087 | 31.0 | 3875 | 0.6293 | 0.7977 | 0.3075 | 1.7781 | 0.7977 | 0.7999 | 0.0799 | 0.0661 |
0.0793 | 32.0 | 4000 | 0.6273 | 0.7973 | 0.3076 | 1.6439 | 0.7973 | 0.7976 | 0.0782 | 0.0695 |
0.0793 | 33.0 | 4125 | 0.6320 | 0.7933 | 0.3123 | 1.6486 | 0.7932 | 0.7954 | 0.0899 | 0.0679 |
0.0793 | 34.0 | 4250 | 0.6345 | 0.79 | 0.3154 | 1.6402 | 0.79 | 0.7903 | 0.0922 | 0.0675 |
0.0793 | 35.0 | 4375 | 0.6209 | 0.793 | 0.3098 | 1.6026 | 0.793 | 0.7943 | 0.0863 | 0.0630 |
0.0733 | 36.0 | 4500 | 0.6187 | 0.7947 | 0.3076 | 1.6282 | 0.7947 | 0.7967 | 0.0880 | 0.0666 |
0.0733 | 37.0 | 4625 | 0.6146 | 0.7957 | 0.3051 | 1.6186 | 0.7957 | 0.7971 | 0.0885 | 0.0623 |
0.0733 | 38.0 | 4750 | 0.6169 | 0.7983 | 0.3062 | 1.6182 | 0.7983 | 0.7996 | 0.0835 | 0.0650 |
0.0733 | 39.0 | 4875 | 0.6180 | 0.7953 | 0.3074 | 1.6241 | 0.7953 | 0.7975 | 0.0889 | 0.0655 |
0.0693 | 40.0 | 5000 | 0.6204 | 0.7977 | 0.3069 | 1.6048 | 0.7977 | 0.7987 | 0.0824 | 0.0659 |
0.0693 | 41.0 | 5125 | 0.6140 | 0.7967 | 0.3055 | 1.6065 | 0.7967 | 0.7986 | 0.0911 | 0.0662 |
0.0693 | 42.0 | 5250 | 0.6162 | 0.7957 | 0.3062 | 1.6182 | 0.7957 | 0.7971 | 0.0883 | 0.0655 |
0.0693 | 43.0 | 5375 | 0.6169 | 0.796 | 0.3058 | 1.6212 | 0.796 | 0.7976 | 0.0879 | 0.0662 |
0.0673 | 44.0 | 5500 | 0.6173 | 0.7973 | 0.3063 | 1.6161 | 0.7973 | 0.7990 | 0.0877 | 0.0666 |
0.0673 | 45.0 | 5625 | 0.6193 | 0.797 | 0.3070 | 1.6151 | 0.797 | 0.7986 | 0.0881 | 0.0678 |
0.0673 | 46.0 | 5750 | 0.6209 | 0.7963 | 0.3076 | 1.6211 | 0.7963 | 0.7979 | 0.0894 | 0.0678 |
0.0673 | 47.0 | 5875 | 0.6211 | 0.7977 | 0.3075 | 1.6284 | 0.7977 | 0.7993 | 0.0905 | 0.0691 |
0.0662 | 48.0 | 6000 | 0.6206 | 0.7967 | 0.3072 | 1.6289 | 0.7967 | 0.7983 | 0.0892 | 0.0673 |
0.0662 | 49.0 | 6125 | 0.6213 | 0.7965 | 0.3075 | 1.6262 | 0.7965 | 0.7980 | 0.0886 | 0.0684 |
0.0662 | 50.0 | 6250 | 0.6215 | 0.7963 | 0.3076 | 1.6291 | 0.7963 | 0.7978 | 0.0919 | 0.0682 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2