<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
cdip-small_rvl_cdip-NK1000_kd_CEKD_t2.5_a0.5
This model is a fine-tuned version of WinKawaks/vit-small-patch16-224 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.4315
- Accuracy: 0.8522
- Brier Loss: 0.2145
- Nll: 1.3474
- F1 Micro: 0.8522
- F1 Macro: 0.8535
- Ece: 0.0573
- Aurc: 0.0300
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 96
- eval_batch_size: 96
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll | F1 Micro | F1 Macro | Ece | Aurc |
---|---|---|---|---|---|---|---|---|---|---|
No log | 1.0 | 167 | 1.6705 | 0.6378 | 0.4837 | 2.4248 | 0.6378 | 0.6323 | 0.0655 | 0.1457 |
No log | 2.0 | 334 | 1.1423 | 0.7322 | 0.3740 | 1.9847 | 0.7322 | 0.7285 | 0.0695 | 0.0846 |
1.7909 | 3.0 | 501 | 0.9082 | 0.7682 | 0.3248 | 1.7674 | 0.7682 | 0.7676 | 0.0620 | 0.0642 |
1.7909 | 4.0 | 668 | 0.8494 | 0.7865 | 0.3082 | 1.7306 | 0.7865 | 0.7904 | 0.0665 | 0.0560 |
1.7909 | 5.0 | 835 | 0.7837 | 0.798 | 0.2988 | 1.6072 | 0.798 | 0.7953 | 0.0729 | 0.0553 |
0.4994 | 6.0 | 1002 | 0.6867 | 0.804 | 0.2862 | 1.5014 | 0.804 | 0.8059 | 0.0794 | 0.0471 |
0.4994 | 7.0 | 1169 | 0.7037 | 0.8157 | 0.2797 | 1.5533 | 0.8157 | 0.8178 | 0.0807 | 0.0478 |
0.4994 | 8.0 | 1336 | 0.6709 | 0.8163 | 0.2756 | 1.5297 | 0.8163 | 0.8166 | 0.0728 | 0.0478 |
0.2478 | 9.0 | 1503 | 0.6132 | 0.825 | 0.2576 | 1.4349 | 0.825 | 0.8247 | 0.0728 | 0.0398 |
0.2478 | 10.0 | 1670 | 0.6389 | 0.8235 | 0.2671 | 1.4455 | 0.8235 | 0.8266 | 0.0746 | 0.0419 |
0.2478 | 11.0 | 1837 | 0.6043 | 0.8257 | 0.2585 | 1.4609 | 0.8257 | 0.8293 | 0.0752 | 0.0403 |
0.1683 | 12.0 | 2004 | 0.5639 | 0.8327 | 0.2457 | 1.4470 | 0.8327 | 0.8350 | 0.0676 | 0.0375 |
0.1683 | 13.0 | 2171 | 0.5665 | 0.8317 | 0.2508 | 1.4054 | 0.8317 | 0.8324 | 0.0731 | 0.0388 |
0.1683 | 14.0 | 2338 | 0.5505 | 0.8403 | 0.2427 | 1.4059 | 0.8403 | 0.8408 | 0.0649 | 0.0377 |
0.131 | 15.0 | 2505 | 0.5321 | 0.836 | 0.2428 | 1.4078 | 0.836 | 0.8372 | 0.0684 | 0.0365 |
0.131 | 16.0 | 2672 | 0.5161 | 0.8373 | 0.2383 | 1.3900 | 0.8373 | 0.8373 | 0.0711 | 0.0368 |
0.131 | 17.0 | 2839 | 0.5177 | 0.8403 | 0.2371 | 1.3828 | 0.8403 | 0.8413 | 0.0633 | 0.0354 |
0.1071 | 18.0 | 3006 | 0.5113 | 0.8407 | 0.2377 | 1.3832 | 0.8407 | 0.8432 | 0.0718 | 0.0343 |
0.1071 | 19.0 | 3173 | 0.4949 | 0.8415 | 0.2332 | 1.3767 | 0.8415 | 0.8428 | 0.0667 | 0.0338 |
0.1071 | 20.0 | 3340 | 0.4857 | 0.848 | 0.2271 | 1.3664 | 0.848 | 0.8492 | 0.0615 | 0.0338 |
0.0877 | 21.0 | 3507 | 0.4812 | 0.847 | 0.2283 | 1.3360 | 0.847 | 0.8478 | 0.0602 | 0.0346 |
0.0877 | 22.0 | 3674 | 0.4715 | 0.8495 | 0.2243 | 1.3761 | 0.8495 | 0.8506 | 0.0560 | 0.0320 |
0.0877 | 23.0 | 3841 | 0.4622 | 0.8508 | 0.2206 | 1.3584 | 0.8508 | 0.8515 | 0.0557 | 0.0323 |
0.0694 | 24.0 | 4008 | 0.4432 | 0.8515 | 0.2167 | 1.3653 | 0.8515 | 0.8531 | 0.0555 | 0.0309 |
0.0694 | 25.0 | 4175 | 0.4467 | 0.8498 | 0.2193 | 1.3499 | 0.8498 | 0.8512 | 0.0581 | 0.0309 |
0.0694 | 26.0 | 4342 | 0.4412 | 0.8545 | 0.2162 | 1.3535 | 0.8545 | 0.8560 | 0.0534 | 0.0306 |
0.0586 | 27.0 | 4509 | 0.4402 | 0.8498 | 0.2180 | 1.3390 | 0.8498 | 0.8510 | 0.0597 | 0.0309 |
0.0586 | 28.0 | 4676 | 0.4408 | 0.8522 | 0.2174 | 1.3568 | 0.8522 | 0.8536 | 0.0576 | 0.0306 |
0.0586 | 29.0 | 4843 | 0.4391 | 0.851 | 0.2168 | 1.3429 | 0.851 | 0.8523 | 0.0585 | 0.0305 |
0.0549 | 30.0 | 5010 | 0.4371 | 0.853 | 0.2160 | 1.3389 | 0.853 | 0.8543 | 0.0573 | 0.0303 |
0.0549 | 31.0 | 5177 | 0.4382 | 0.8498 | 0.2168 | 1.3486 | 0.8498 | 0.8513 | 0.0602 | 0.0304 |
0.0549 | 32.0 | 5344 | 0.4372 | 0.853 | 0.2166 | 1.3501 | 0.853 | 0.8540 | 0.0591 | 0.0306 |
0.0527 | 33.0 | 5511 | 0.4379 | 0.852 | 0.2156 | 1.3546 | 0.852 | 0.8531 | 0.0576 | 0.0304 |
0.0527 | 34.0 | 5678 | 0.4353 | 0.8532 | 0.2154 | 1.3381 | 0.8532 | 0.8543 | 0.0574 | 0.0302 |
0.0527 | 35.0 | 5845 | 0.4347 | 0.8525 | 0.2148 | 1.3550 | 0.8525 | 0.8535 | 0.0591 | 0.0304 |
0.0511 | 36.0 | 6012 | 0.4311 | 0.8542 | 0.2141 | 1.3233 | 0.8542 | 0.8552 | 0.0572 | 0.0299 |
0.0511 | 37.0 | 6179 | 0.4323 | 0.852 | 0.2150 | 1.3332 | 0.852 | 0.8532 | 0.0586 | 0.0302 |
0.0511 | 38.0 | 6346 | 0.4321 | 0.8515 | 0.2152 | 1.3382 | 0.8515 | 0.8527 | 0.0583 | 0.0299 |
0.0494 | 39.0 | 6513 | 0.4335 | 0.8495 | 0.2152 | 1.3385 | 0.8495 | 0.8511 | 0.0593 | 0.0303 |
0.0494 | 40.0 | 6680 | 0.4323 | 0.852 | 0.2146 | 1.3603 | 0.852 | 0.8533 | 0.0576 | 0.0299 |
0.0494 | 41.0 | 6847 | 0.4309 | 0.8512 | 0.2143 | 1.3448 | 0.8512 | 0.8525 | 0.0570 | 0.0299 |
0.0477 | 42.0 | 7014 | 0.4327 | 0.8525 | 0.2149 | 1.3439 | 0.8525 | 0.8539 | 0.0580 | 0.0299 |
0.0477 | 43.0 | 7181 | 0.4309 | 0.8532 | 0.2140 | 1.3406 | 0.8532 | 0.8544 | 0.0560 | 0.0299 |
0.0477 | 44.0 | 7348 | 0.4308 | 0.8528 | 0.2141 | 1.3404 | 0.8528 | 0.8540 | 0.0573 | 0.0299 |
0.0466 | 45.0 | 7515 | 0.4317 | 0.8525 | 0.2147 | 1.3402 | 0.8525 | 0.8538 | 0.0580 | 0.0299 |
0.0466 | 46.0 | 7682 | 0.4317 | 0.8535 | 0.2144 | 1.3475 | 0.8535 | 0.8547 | 0.0553 | 0.0298 |
0.0466 | 47.0 | 7849 | 0.4314 | 0.8525 | 0.2143 | 1.3479 | 0.8525 | 0.8537 | 0.0559 | 0.0299 |
0.0465 | 48.0 | 8016 | 0.4314 | 0.8525 | 0.2143 | 1.3479 | 0.8525 | 0.8538 | 0.0559 | 0.0299 |
0.0465 | 49.0 | 8183 | 0.4316 | 0.8528 | 0.2145 | 1.3471 | 0.8528 | 0.8540 | 0.0573 | 0.0299 |
0.0465 | 50.0 | 8350 | 0.4315 | 0.8522 | 0.2145 | 1.3474 | 0.8522 | 0.8535 | 0.0573 | 0.0300 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2