<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
Variome_0.0001_0404_ES6
This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.0656
- Precision: 0.6474
- Recall: 0.5691
- F1: 0.6057
- Accuracy: 0.9854
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 2000
Training results
Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
---|---|---|---|---|---|---|---|
0.4714 | 0.13 | 25 | 0.1791 | 0.0 | 0.0 | 0.0 | 0.9759 |
0.1531 | 0.26 | 50 | 0.1271 | 0.0 | 0.0 | 0.0 | 0.9759 |
0.1288 | 0.39 | 75 | 0.1253 | 0.2032 | 0.0864 | 0.1212 | 0.9770 |
0.1253 | 0.52 | 100 | 0.1143 | 0.425 | 0.0163 | 0.0314 | 0.9762 |
0.101 | 0.65 | 125 | 0.1124 | 0.3438 | 0.1574 | 0.2159 | 0.9780 |
0.1062 | 0.79 | 150 | 0.1014 | 0.3569 | 0.2860 | 0.3175 | 0.9793 |
0.092 | 0.92 | 175 | 0.0986 | 0.4274 | 0.3081 | 0.3581 | 0.9805 |
0.0702 | 1.05 | 200 | 0.0942 | 0.5349 | 0.3234 | 0.4031 | 0.9810 |
0.0881 | 1.18 | 225 | 0.1009 | 0.3379 | 0.3580 | 0.3476 | 0.9765 |
0.0866 | 1.31 | 250 | 0.0780 | 0.5453 | 0.4155 | 0.4717 | 0.9830 |
0.0751 | 1.44 | 275 | 0.0749 | 0.5157 | 0.5038 | 0.5097 | 0.9830 |
0.0679 | 1.57 | 300 | 0.0732 | 0.5401 | 0.5432 | 0.5416 | 0.9835 |
0.0625 | 1.7 | 325 | 0.0728 | 0.6411 | 0.4491 | 0.5282 | 0.9842 |
0.065 | 1.83 | 350 | 0.0737 | 0.6115 | 0.4683 | 0.5304 | 0.9839 |
0.051 | 1.96 | 375 | 0.0720 | 0.5928 | 0.5240 | 0.5563 | 0.9840 |
0.0503 | 2.09 | 400 | 0.0669 | 0.6124 | 0.5489 | 0.5789 | 0.9854 |
0.0464 | 2.23 | 425 | 0.0718 | 0.625 | 0.5278 | 0.5723 | 0.9847 |
0.0528 | 2.36 | 450 | 0.0763 | 0.6628 | 0.4395 | 0.5286 | 0.9838 |
0.0473 | 2.49 | 475 | 0.0706 | 0.6109 | 0.5393 | 0.5729 | 0.9848 |
0.0542 | 2.62 | 500 | 0.0657 | 0.6159 | 0.5432 | 0.5773 | 0.9845 |
0.0508 | 2.75 | 525 | 0.0686 | 0.6515 | 0.5653 | 0.6053 | 0.9848 |
0.0476 | 2.88 | 550 | 0.0631 | 0.6857 | 0.5528 | 0.6121 | 0.9860 |
0.049 | 3.01 | 575 | 0.0689 | 0.6008 | 0.5720 | 0.5860 | 0.9842 |
0.0378 | 3.14 | 600 | 0.0720 | 0.6674 | 0.5605 | 0.6093 | 0.9854 |
0.0417 | 3.27 | 625 | 0.0695 | 0.6248 | 0.6104 | 0.6175 | 0.9852 |
0.0306 | 3.4 | 650 | 0.0723 | 0.6780 | 0.5537 | 0.6096 | 0.9860 |
0.0341 | 3.53 | 675 | 0.0664 | 0.6651 | 0.5547 | 0.6049 | 0.9862 |
0.0392 | 3.66 | 700 | 0.0656 | 0.6474 | 0.5691 | 0.6057 | 0.9854 |
Framework versions
- Transformers 4.27.4
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.2