<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
dna_bert_3_1000seq-finetuned
This model is a fine-tuned version of armheb/DNA_bert_3 on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.4684
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
0.8607 | 1.0 | 62 | 0.6257 |
0.6177 | 2.0 | 124 | 0.6120 |
0.6098 | 3.0 | 186 | 0.6062 |
0.604 | 4.0 | 248 | 0.6052 |
0.5999 | 5.0 | 310 | 0.6040 |
0.5982 | 6.0 | 372 | 0.5996 |
0.5985 | 7.0 | 434 | 0.5985 |
0.5956 | 8.0 | 496 | 0.5968 |
0.5936 | 9.0 | 558 | 0.5950 |
0.5908 | 10.0 | 620 | 0.5941 |
0.5904 | 11.0 | 682 | 0.5932 |
0.59 | 12.0 | 744 | 0.5917 |
0.5877 | 13.0 | 806 | 0.5921 |
0.5847 | 14.0 | 868 | 0.5903 |
0.5831 | 15.0 | 930 | 0.5887 |
0.5852 | 16.0 | 992 | 0.5878 |
0.5805 | 17.0 | 1054 | 0.5872 |
0.5795 | 18.0 | 1116 | 0.5853 |
0.5754 | 19.0 | 1178 | 0.5869 |
0.5757 | 20.0 | 1240 | 0.5839 |
0.5722 | 21.0 | 1302 | 0.5831 |
0.5693 | 22.0 | 1364 | 0.5811 |
0.5667 | 23.0 | 1426 | 0.5802 |
0.5652 | 24.0 | 1488 | 0.5775 |
0.5608 | 25.0 | 1550 | 0.5788 |
0.5591 | 26.0 | 1612 | 0.5724 |
0.5538 | 27.0 | 1674 | 0.5736 |
0.552 | 28.0 | 1736 | 0.5689 |
0.5483 | 29.0 | 1798 | 0.5689 |
0.5442 | 30.0 | 1860 | 0.5671 |
0.5405 | 31.0 | 1922 | 0.5658 |
0.537 | 32.0 | 1984 | 0.5605 |
0.5349 | 33.0 | 2046 | 0.5575 |
0.5275 | 34.0 | 2108 | 0.5569 |
0.5227 | 35.0 | 2170 | 0.5537 |
0.52 | 36.0 | 2232 | 0.5509 |
0.5173 | 37.0 | 2294 | 0.5504 |
0.5123 | 38.0 | 2356 | 0.5435 |
0.5088 | 39.0 | 2418 | 0.5472 |
0.5037 | 40.0 | 2480 | 0.5383 |
0.501 | 41.0 | 2542 | 0.5379 |
0.4931 | 42.0 | 2604 | 0.5365 |
0.4923 | 43.0 | 2666 | 0.5328 |
0.4879 | 44.0 | 2728 | 0.5301 |
0.482 | 45.0 | 2790 | 0.5295 |
0.4805 | 46.0 | 2852 | 0.5261 |
0.4772 | 47.0 | 2914 | 0.5221 |
0.4738 | 48.0 | 2976 | 0.5234 |
0.4674 | 49.0 | 3038 | 0.5210 |
0.4646 | 50.0 | 3100 | 0.5169 |
0.4621 | 51.0 | 3162 | 0.5142 |
0.4574 | 52.0 | 3224 | 0.5129 |
0.4552 | 53.0 | 3286 | 0.5127 |
0.4539 | 54.0 | 3348 | 0.5124 |
0.4506 | 55.0 | 3410 | 0.5076 |
0.4457 | 56.0 | 3472 | 0.5082 |
0.4454 | 57.0 | 3534 | 0.5027 |
0.4398 | 58.0 | 3596 | 0.5019 |
0.4386 | 59.0 | 3658 | 0.4998 |
0.4332 | 60.0 | 3720 | 0.4970 |
0.4277 | 61.0 | 3782 | 0.4995 |
0.4273 | 62.0 | 3844 | 0.4962 |
0.4235 | 63.0 | 3906 | 0.4909 |
0.4201 | 64.0 | 3968 | 0.4913 |
0.4198 | 65.0 | 4030 | 0.4899 |
0.4182 | 66.0 | 4092 | 0.4919 |
0.4157 | 67.0 | 4154 | 0.4902 |
0.4104 | 68.0 | 4216 | 0.4881 |
0.4095 | 69.0 | 4278 | 0.4881 |
0.4077 | 70.0 | 4340 | 0.4861 |
0.4064 | 71.0 | 4402 | 0.4868 |
0.4041 | 72.0 | 4464 | 0.4826 |
0.4029 | 73.0 | 4526 | 0.4833 |
0.3976 | 74.0 | 4588 | 0.4819 |
0.3997 | 75.0 | 4650 | 0.4809 |
0.3974 | 76.0 | 4712 | 0.4801 |
0.3953 | 77.0 | 4774 | 0.4783 |
0.3938 | 78.0 | 4836 | 0.4775 |
0.3934 | 79.0 | 4898 | 0.4762 |
0.3923 | 80.0 | 4960 | 0.4742 |
0.3893 | 81.0 | 5022 | 0.4742 |
0.3909 | 82.0 | 5084 | 0.4740 |
0.3856 | 83.0 | 5146 | 0.4739 |
0.3904 | 84.0 | 5208 | 0.4740 |
0.3883 | 85.0 | 5270 | 0.4701 |
0.3865 | 86.0 | 5332 | 0.4727 |
0.3809 | 87.0 | 5394 | 0.4736 |
0.3853 | 88.0 | 5456 | 0.4704 |
0.3821 | 89.0 | 5518 | 0.4704 |
0.3809 | 90.0 | 5580 | 0.4701 |
0.3814 | 91.0 | 5642 | 0.4698 |
0.3795 | 92.0 | 5704 | 0.4702 |
0.3804 | 93.0 | 5766 | 0.4692 |
0.377 | 94.0 | 5828 | 0.4683 |
0.3812 | 95.0 | 5890 | 0.4692 |
0.3806 | 96.0 | 5952 | 0.4683 |
0.3745 | 97.0 | 6014 | 0.4690 |
0.3825 | 98.0 | 6076 | 0.4684 |
0.374 | 99.0 | 6138 | 0.4687 |
0.3795 | 100.0 | 6200 | 0.4684 |
Framework versions
- Transformers 4.21.1
- Pytorch 1.12.0+cu113
- Datasets 2.4.0
- Tokenizers 0.12.1