<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
vit-base_rvl-cdip-tiny_rvl_cdip-NK1000_hint_rand
This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the None dataset. It achieves the following results on the evaluation set:
- Loss: 75.5808
- Accuracy: 0.583
- Brier Loss: 0.7311
- Nll: 3.9633
- F1 Micro: 0.583
- F1 Macro: 0.5838
- Ece: 0.3399
- Aurc: 0.2128
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | Brier Loss | Nll | F1 Micro | F1 Macro | Ece | Aurc |
|---|---|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 250 | 78.0119 | 0.1285 | 0.9098 | 6.7342 | 0.1285 | 0.0748 | 0.0496 | 0.7634 |
| 77.7969 | 2.0 | 500 | 77.3633 | 0.1595 | 0.8985 | 5.2942 | 0.1595 | 0.1038 | 0.0509 | 0.7216 |
| 77.7969 | 3.0 | 750 | 76.6773 | 0.2545 | 0.8551 | 3.9015 | 0.2545 | 0.2006 | 0.0741 | 0.5967 |
| 76.735 | 4.0 | 1000 | 76.1721 | 0.312 | 0.8123 | 3.4141 | 0.312 | 0.2785 | 0.0855 | 0.5018 |
| 76.735 | 5.0 | 1250 | 76.0027 | 0.3703 | 0.7573 | 3.2539 | 0.3703 | 0.3299 | 0.0764 | 0.4161 |
| 75.8262 | 6.0 | 1500 | 76.3256 | 0.4143 | 0.7290 | 3.1129 | 0.4143 | 0.3995 | 0.0835 | 0.3792 |
| 75.8262 | 7.0 | 1750 | 75.5753 | 0.4575 | 0.6838 | 2.8940 | 0.4575 | 0.4421 | 0.0595 | 0.3262 |
| 75.3656 | 8.0 | 2000 | 75.2875 | 0.475 | 0.6554 | 2.7996 | 0.4750 | 0.4596 | 0.0715 | 0.2976 |
| 75.3656 | 9.0 | 2250 | 75.3849 | 0.4833 | 0.6446 | 2.7232 | 0.4833 | 0.4523 | 0.0651 | 0.2885 |
| 75.0748 | 10.0 | 2500 | 75.3431 | 0.5172 | 0.6173 | 2.6664 | 0.5172 | 0.4905 | 0.0563 | 0.2606 |
| 75.0748 | 11.0 | 2750 | 75.0478 | 0.5357 | 0.5982 | 2.7014 | 0.5357 | 0.5207 | 0.0550 | 0.2384 |
| 74.821 | 12.0 | 3000 | 75.1324 | 0.5325 | 0.5973 | 2.6161 | 0.5325 | 0.5202 | 0.0569 | 0.2402 |
| 74.821 | 13.0 | 3250 | 75.0049 | 0.528 | 0.5996 | 2.6859 | 0.528 | 0.5157 | 0.0657 | 0.2408 |
| 74.613 | 14.0 | 3500 | 74.8702 | 0.5453 | 0.5881 | 2.7150 | 0.5453 | 0.5455 | 0.0661 | 0.2302 |
| 74.613 | 15.0 | 3750 | 74.8427 | 0.5595 | 0.5697 | 2.5605 | 0.5595 | 0.5479 | 0.0765 | 0.2117 |
| 74.421 | 16.0 | 4000 | 74.9157 | 0.5503 | 0.5829 | 2.7215 | 0.5503 | 0.5524 | 0.0765 | 0.2219 |
| 74.421 | 17.0 | 4250 | 74.9051 | 0.5633 | 0.5816 | 2.6715 | 0.5633 | 0.5577 | 0.0924 | 0.2186 |
| 74.2453 | 18.0 | 4500 | 74.9910 | 0.5733 | 0.5722 | 2.6963 | 0.5733 | 0.5717 | 0.0930 | 0.2107 |
| 74.2453 | 19.0 | 4750 | 74.8632 | 0.5575 | 0.5892 | 2.6981 | 0.5575 | 0.5549 | 0.1073 | 0.2198 |
| 74.0712 | 20.0 | 5000 | 74.8128 | 0.5757 | 0.5794 | 2.7227 | 0.5757 | 0.5697 | 0.1235 | 0.2083 |
| 74.0712 | 21.0 | 5250 | 74.7545 | 0.575 | 0.5794 | 2.7000 | 0.575 | 0.5700 | 0.1372 | 0.2015 |
| 73.9033 | 22.0 | 5500 | 74.7493 | 0.5737 | 0.5841 | 2.7996 | 0.5737 | 0.5806 | 0.1341 | 0.2073 |
| 73.9033 | 23.0 | 5750 | 74.7641 | 0.582 | 0.5831 | 2.7846 | 0.582 | 0.5780 | 0.1576 | 0.1985 |
| 73.7364 | 24.0 | 6000 | 74.8125 | 0.5807 | 0.5944 | 2.8725 | 0.5807 | 0.5767 | 0.1719 | 0.2015 |
| 73.7364 | 25.0 | 6250 | 74.9721 | 0.573 | 0.6132 | 2.9232 | 0.573 | 0.5734 | 0.1920 | 0.2086 |
| 73.5899 | 26.0 | 6500 | 74.8675 | 0.5823 | 0.6127 | 2.9200 | 0.5823 | 0.5788 | 0.1969 | 0.2059 |
| 73.5899 | 27.0 | 6750 | 74.9213 | 0.5723 | 0.6234 | 3.0482 | 0.5723 | 0.5717 | 0.2138 | 0.2085 |
| 73.4419 | 28.0 | 7000 | 74.9436 | 0.5815 | 0.6324 | 3.0789 | 0.5815 | 0.5803 | 0.2223 | 0.2058 |
| 73.4419 | 29.0 | 7250 | 74.8826 | 0.5747 | 0.6408 | 3.1380 | 0.5747 | 0.5711 | 0.2428 | 0.2044 |
| 73.3198 | 30.0 | 7500 | 75.0310 | 0.5633 | 0.6722 | 3.2517 | 0.5633 | 0.5639 | 0.2571 | 0.2226 |
| 73.3198 | 31.0 | 7750 | 75.0300 | 0.5577 | 0.6795 | 3.3520 | 0.5577 | 0.5627 | 0.2611 | 0.2255 |
| 73.2086 | 32.0 | 8000 | 74.9569 | 0.5793 | 0.6614 | 3.3345 | 0.5793 | 0.5829 | 0.2623 | 0.2070 |
| 73.2086 | 33.0 | 8250 | 75.1474 | 0.5655 | 0.6902 | 3.5319 | 0.5655 | 0.5656 | 0.2780 | 0.2260 |
| 73.1102 | 34.0 | 8500 | 75.1176 | 0.5697 | 0.6926 | 3.5011 | 0.5697 | 0.5685 | 0.2891 | 0.2127 |
| 73.1102 | 35.0 | 8750 | 75.2834 | 0.5673 | 0.7085 | 3.7150 | 0.5673 | 0.5688 | 0.2945 | 0.2210 |
| 73.0239 | 36.0 | 9000 | 75.2426 | 0.566 | 0.7101 | 3.6822 | 0.566 | 0.5679 | 0.3029 | 0.2200 |
| 73.0239 | 37.0 | 9250 | 75.3049 | 0.5743 | 0.7082 | 3.6300 | 0.5743 | 0.5758 | 0.3044 | 0.2185 |
| 72.9631 | 38.0 | 9500 | 75.3404 | 0.5695 | 0.7220 | 3.7386 | 0.5695 | 0.5741 | 0.3177 | 0.2210 |
| 72.9631 | 39.0 | 9750 | 75.4376 | 0.5775 | 0.7181 | 3.8412 | 0.5775 | 0.5784 | 0.3148 | 0.2191 |
| 72.9028 | 40.0 | 10000 | 75.4664 | 0.5777 | 0.7178 | 3.9272 | 0.5777 | 0.5775 | 0.3178 | 0.2233 |
| 72.9028 | 41.0 | 10250 | 75.5305 | 0.5737 | 0.7279 | 3.8240 | 0.5737 | 0.5761 | 0.3271 | 0.2215 |
| 72.8505 | 42.0 | 10500 | 75.4606 | 0.5783 | 0.7225 | 3.8401 | 0.5783 | 0.5805 | 0.3261 | 0.2156 |
| 72.8505 | 43.0 | 10750 | 75.5084 | 0.5793 | 0.7242 | 3.8552 | 0.5793 | 0.5791 | 0.3308 | 0.2115 |
| 72.8091 | 44.0 | 11000 | 75.4797 | 0.5817 | 0.7256 | 3.8946 | 0.5817 | 0.5825 | 0.3340 | 0.2112 |
| 72.8091 | 45.0 | 11250 | 75.5695 | 0.5793 | 0.7297 | 3.9742 | 0.5793 | 0.5809 | 0.3379 | 0.2150 |
| 72.7801 | 46.0 | 11500 | 75.5592 | 0.5807 | 0.7331 | 3.9445 | 0.5807 | 0.5830 | 0.3378 | 0.2151 |
| 72.7801 | 47.0 | 11750 | 75.5976 | 0.5833 | 0.7303 | 3.9669 | 0.5833 | 0.5840 | 0.3380 | 0.2145 |
| 72.7606 | 48.0 | 12000 | 75.5952 | 0.5833 | 0.7320 | 3.9813 | 0.5833 | 0.5847 | 0.3380 | 0.2148 |
| 72.7606 | 49.0 | 12250 | 75.5621 | 0.5843 | 0.7309 | 3.9491 | 0.5843 | 0.5851 | 0.3385 | 0.2127 |
| 72.7486 | 50.0 | 12500 | 75.5808 | 0.583 | 0.7311 | 3.9633 | 0.583 | 0.5838 | 0.3399 | 0.2128 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.1.post200
- Datasets 2.9.0
- Tokenizers 0.13.2
