<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
cdip-tiny_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5
This model is a fine-tuned version of WinKawaks/vit-tiny-patch16-224 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.5622
- Accuracy: 0.8255
- Brier Loss: 0.2585
- Nll: 1.9229
- F1 Micro: 0.8255
- F1 Macro: 0.8273
- Ece: 0.0661
- Aurc: 0.0421
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll | F1 Micro | F1 Macro | Ece | Aurc |
---|---|---|---|---|---|---|---|---|---|---|
No log | 1.0 | 125 | 2.1936 | 0.55 | 0.5790 | 2.6092 | 0.55 | 0.5380 | 0.0723 | 0.2136 |
No log | 2.0 | 250 | 1.5136 | 0.666 | 0.4551 | 2.2832 | 0.666 | 0.6632 | 0.0686 | 0.1282 |
No log | 3.0 | 375 | 1.2758 | 0.6993 | 0.4127 | 2.2483 | 0.6993 | 0.6928 | 0.0836 | 0.1026 |
1.9726 | 4.0 | 500 | 1.1382 | 0.726 | 0.3794 | 2.0849 | 0.726 | 0.7288 | 0.0725 | 0.0878 |
1.9726 | 5.0 | 625 | 1.0625 | 0.7542 | 0.3523 | 2.1178 | 0.7542 | 0.7426 | 0.0755 | 0.0758 |
1.9726 | 6.0 | 750 | 0.8657 | 0.7775 | 0.3184 | 1.9516 | 0.7775 | 0.7783 | 0.0736 | 0.0618 |
1.9726 | 7.0 | 875 | 0.8285 | 0.7887 | 0.3089 | 2.0439 | 0.7887 | 0.7867 | 0.0724 | 0.0586 |
0.5092 | 8.0 | 1000 | 0.7988 | 0.7925 | 0.3056 | 2.0106 | 0.7925 | 0.7953 | 0.0700 | 0.0572 |
0.5092 | 9.0 | 1125 | 0.7783 | 0.7925 | 0.3060 | 1.9710 | 0.7925 | 0.7914 | 0.0822 | 0.0573 |
0.5092 | 10.0 | 1250 | 0.7640 | 0.796 | 0.3007 | 1.9819 | 0.796 | 0.7996 | 0.0801 | 0.0536 |
0.5092 | 11.0 | 1375 | 0.7705 | 0.7913 | 0.3048 | 1.9588 | 0.7913 | 0.7952 | 0.0857 | 0.0559 |
0.2071 | 12.0 | 1500 | 0.7328 | 0.8015 | 0.2937 | 1.9484 | 0.8015 | 0.8017 | 0.0760 | 0.0537 |
0.2071 | 13.0 | 1625 | 0.6946 | 0.811 | 0.2881 | 1.9173 | 0.811 | 0.8125 | 0.0824 | 0.0507 |
0.2071 | 14.0 | 1750 | 0.6902 | 0.8053 | 0.2880 | 1.9154 | 0.8053 | 0.8068 | 0.0738 | 0.0502 |
0.2071 | 15.0 | 1875 | 0.6756 | 0.8083 | 0.2840 | 1.9317 | 0.8083 | 0.8078 | 0.0761 | 0.0487 |
0.1424 | 16.0 | 2000 | 0.6684 | 0.8067 | 0.2852 | 1.9192 | 0.8067 | 0.8073 | 0.0765 | 0.0507 |
0.1424 | 17.0 | 2125 | 0.6548 | 0.8095 | 0.2816 | 1.9398 | 0.8095 | 0.8110 | 0.0758 | 0.0472 |
0.1424 | 18.0 | 2250 | 0.6477 | 0.8117 | 0.2762 | 1.9054 | 0.8117 | 0.8140 | 0.0759 | 0.0464 |
0.1424 | 19.0 | 2375 | 0.6423 | 0.8145 | 0.2794 | 1.9081 | 0.8145 | 0.8148 | 0.0774 | 0.0478 |
0.1102 | 20.0 | 2500 | 0.6312 | 0.8103 | 0.2771 | 1.9581 | 0.8103 | 0.8125 | 0.0746 | 0.0454 |
0.1102 | 21.0 | 2625 | 0.6299 | 0.8133 | 0.2720 | 1.9275 | 0.8133 | 0.8132 | 0.0758 | 0.0466 |
0.1102 | 22.0 | 2750 | 0.6148 | 0.8197 | 0.2691 | 1.9463 | 0.8197 | 0.8223 | 0.0681 | 0.0447 |
0.1102 | 23.0 | 2875 | 0.6132 | 0.8187 | 0.2700 | 1.9301 | 0.8187 | 0.8200 | 0.0691 | 0.0451 |
0.0931 | 24.0 | 3000 | 0.5995 | 0.8245 | 0.2649 | 1.9173 | 0.8245 | 0.8251 | 0.0640 | 0.0444 |
0.0931 | 25.0 | 3125 | 0.6020 | 0.8177 | 0.2697 | 1.9205 | 0.8178 | 0.8201 | 0.0723 | 0.0440 |
0.0931 | 26.0 | 3250 | 0.5914 | 0.8247 | 0.2617 | 1.9385 | 0.8247 | 0.8264 | 0.0667 | 0.0428 |
0.0931 | 27.0 | 3375 | 0.5833 | 0.822 | 0.2621 | 1.9390 | 0.822 | 0.8228 | 0.0658 | 0.0429 |
0.0789 | 28.0 | 3500 | 0.5884 | 0.8247 | 0.2626 | 1.9400 | 0.8247 | 0.8259 | 0.0619 | 0.0435 |
0.0789 | 29.0 | 3625 | 0.5771 | 0.8285 | 0.2568 | 1.9252 | 0.8285 | 0.8313 | 0.0612 | 0.0413 |
0.0789 | 30.0 | 3750 | 0.5815 | 0.823 | 0.2628 | 1.9413 | 0.823 | 0.8236 | 0.0676 | 0.0433 |
0.0789 | 31.0 | 3875 | 0.5789 | 0.8205 | 0.2617 | 1.9209 | 0.8205 | 0.8219 | 0.0667 | 0.0431 |
0.0686 | 32.0 | 4000 | 0.5775 | 0.8247 | 0.2616 | 1.9045 | 0.8247 | 0.8265 | 0.0674 | 0.0428 |
0.0686 | 33.0 | 4125 | 0.5744 | 0.827 | 0.2603 | 1.9088 | 0.827 | 0.8275 | 0.0656 | 0.0420 |
0.0686 | 34.0 | 4250 | 0.5685 | 0.824 | 0.2607 | 1.9372 | 0.824 | 0.8264 | 0.0647 | 0.0421 |
0.0686 | 35.0 | 4375 | 0.5649 | 0.8255 | 0.2584 | 1.9375 | 0.8255 | 0.8274 | 0.0694 | 0.0419 |
0.0596 | 36.0 | 4500 | 0.5629 | 0.8263 | 0.2574 | 1.9304 | 0.8263 | 0.8283 | 0.0651 | 0.0415 |
0.0596 | 37.0 | 4625 | 0.5622 | 0.8237 | 0.2579 | 1.9228 | 0.8237 | 0.8254 | 0.0644 | 0.0419 |
0.0596 | 38.0 | 4750 | 0.5623 | 0.8257 | 0.2579 | 1.9310 | 0.8257 | 0.8277 | 0.0650 | 0.0418 |
0.0596 | 39.0 | 4875 | 0.5625 | 0.827 | 0.2579 | 1.9311 | 0.827 | 0.8286 | 0.0668 | 0.0418 |
0.0538 | 40.0 | 5000 | 0.5633 | 0.8247 | 0.2590 | 1.9264 | 0.8247 | 0.8264 | 0.0671 | 0.0424 |
0.0538 | 41.0 | 5125 | 0.5607 | 0.8257 | 0.2575 | 1.9239 | 0.8257 | 0.8275 | 0.0621 | 0.0417 |
0.0538 | 42.0 | 5250 | 0.5605 | 0.8263 | 0.2569 | 1.9305 | 0.8263 | 0.8279 | 0.0620 | 0.0418 |
0.0538 | 43.0 | 5375 | 0.5613 | 0.8255 | 0.2581 | 1.9295 | 0.8255 | 0.8272 | 0.0672 | 0.0418 |
0.0512 | 44.0 | 5500 | 0.5616 | 0.8255 | 0.2581 | 1.9235 | 0.8255 | 0.8273 | 0.0636 | 0.0419 |
0.0512 | 45.0 | 5625 | 0.5624 | 0.8253 | 0.2585 | 1.9206 | 0.8253 | 0.8270 | 0.0646 | 0.0422 |
0.0512 | 46.0 | 5750 | 0.5622 | 0.8257 | 0.2582 | 1.9283 | 0.8257 | 0.8273 | 0.0647 | 0.0420 |
0.0512 | 47.0 | 5875 | 0.5616 | 0.825 | 0.2581 | 1.9283 | 0.825 | 0.8267 | 0.0640 | 0.0420 |
0.05 | 48.0 | 6000 | 0.5620 | 0.8257 | 0.2584 | 1.9262 | 0.8257 | 0.8275 | 0.0644 | 0.0421 |
0.05 | 49.0 | 6125 | 0.5622 | 0.8253 | 0.2585 | 1.9257 | 0.8253 | 0.8270 | 0.0630 | 0.0421 |
0.05 | 50.0 | 6250 | 0.5622 | 0.8255 | 0.2585 | 1.9229 | 0.8255 | 0.8273 | 0.0661 | 0.0421 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2