<!-- This model card has been generated automatically according to the information Keras had access to. You should probably proofread and complete it, then remove this comment. -->
specializedMedBERT
This model is a fine-tuned version of smanjil/German-MedBERT on an unknown dataset. It achieves the following results on the evaluation set:
- Train Loss: 1.5315
- Train Sparse Categorical Accuracy: 0.8651
- Validation Loss: 9.0076
- Validation Sparse Categorical Accuracy: 0.0
- Epoch: 99
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'learning_rate': {'class_name': 'ExponentialDecay', 'config': {'initial_learning_rate': 5e-05, 'decay_steps': 10000, 'decay_rate': 0.9, 'staircase': False, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32
Training results
Train Loss | Train Sparse Categorical Accuracy | Validation Loss | Validation Sparse Categorical Accuracy | Epoch |
---|---|---|---|---|
5.9919 | 0.0033 | 6.0158 | 0.0 | 0 |
5.9288 | 0.0033 | 6.1132 | 0.0 | 1 |
5.8990 | 0.0033 | 6.2120 | 0.0 | 2 |
5.8696 | 0.0099 | 6.3116 | 0.0 | 3 |
5.8431 | 0.0066 | 6.4143 | 0.0 | 4 |
5.8144 | 0.0066 | 6.5193 | 0.0 | 5 |
5.7883 | 0.0066 | 6.6265 | 0.0 | 6 |
5.7623 | 0.0132 | 6.7340 | 0.0 | 7 |
5.7480 | 0.0164 | 6.8418 | 0.0 | 8 |
5.7154 | 0.0197 | 6.9496 | 0.0 | 9 |
5.6937 | 0.0164 | 7.0557 | 0.0 | 10 |
5.6707 | 0.0230 | 7.1567 | 0.0 | 11 |
5.6377 | 0.0263 | 7.2541 | 0.0 | 12 |
5.6186 | 0.0362 | 7.3436 | 0.0 | 13 |
5.5964 | 0.0428 | 7.4283 | 0.0 | 14 |
5.5677 | 0.0428 | 7.5049 | 0.0 | 15 |
5.5230 | 0.0428 | 7.5759 | 0.0 | 16 |
5.5032 | 0.0526 | 7.6312 | 0.0 | 17 |
5.4563 | 0.0526 | 7.6620 | 0.0 | 18 |
5.4089 | 0.0625 | 7.6913 | 0.0 | 19 |
5.3804 | 0.0691 | 7.7320 | 0.0 | 20 |
5.3208 | 0.0757 | 7.7632 | 0.0 | 21 |
5.2766 | 0.0855 | 7.7825 | 0.0 | 22 |
5.2238 | 0.0855 | 7.7927 | 0.0 | 23 |
5.1774 | 0.0987 | 7.7880 | 0.0 | 24 |
5.1243 | 0.0954 | 7.8373 | 0.0 | 25 |
5.0660 | 0.1053 | 7.7977 | 0.0 | 26 |
4.9892 | 0.1612 | 7.8100 | 0.0 | 27 |
4.9268 | 0.1678 | 7.8606 | 0.0 | 28 |
4.8715 | 0.1776 | 7.8000 | 0.0 | 29 |
4.8120 | 0.2204 | 7.8334 | 0.0 | 30 |
4.7476 | 0.2303 | 7.8024 | 0.0 | 31 |
4.6914 | 0.2270 | 7.8134 | 0.0 | 32 |
4.6176 | 0.2796 | 7.8448 | 0.0 | 33 |
4.5467 | 0.2928 | 7.8147 | 0.0 | 34 |
4.4950 | 0.3026 | 7.8551 | 0.0 | 35 |
4.4183 | 0.3388 | 7.8377 | 0.0 | 36 |
4.3539 | 0.3421 | 7.8461 | 0.0 | 37 |
4.2908 | 0.3684 | 7.8606 | 0.0 | 38 |
4.2328 | 0.4013 | 7.8715 | 0.0 | 39 |
4.1691 | 0.4112 | 7.8789 | 0.0 | 40 |
4.1010 | 0.4079 | 7.8816 | 0.0 | 41 |
4.0412 | 0.4441 | 7.8609 | 0.0 | 42 |
3.9727 | 0.4474 | 7.8889 | 0.0 | 43 |
3.9329 | 0.4342 | 7.9114 | 0.0 | 44 |
3.8540 | 0.4539 | 7.9427 | 0.0 | 45 |
3.7994 | 0.4539 | 7.9431 | 0.0 | 46 |
3.7598 | 0.5066 | 7.9519 | 0.0 | 47 |
3.6893 | 0.5197 | 7.9339 | 0.0 | 48 |
3.6267 | 0.5066 | 8.0038 | 0.0 | 49 |
3.5809 | 0.4901 | 7.9978 | 0.0 | 50 |
3.4976 | 0.5559 | 8.0268 | 0.0 | 51 |
3.4580 | 0.5329 | 8.0554 | 0.0 | 52 |
3.4000 | 0.5855 | 8.0480 | 0.0 | 53 |
3.3548 | 0.5757 | 8.0821 | 0.0 | 54 |
3.3000 | 0.5921 | 8.1102 | 0.0 | 55 |
3.2411 | 0.6086 | 8.1026 | 0.0 | 56 |
3.2002 | 0.6053 | 8.1572 | 0.0 | 57 |
3.1600 | 0.5888 | 8.1550 | 0.0 | 58 |
3.0738 | 0.6118 | 8.1648 | 0.0 | 59 |
3.0213 | 0.6842 | 8.1901 | 0.0 | 60 |
2.9889 | 0.6283 | 8.2164 | 0.0 | 61 |
2.9368 | 0.6414 | 8.2512 | 0.0 | 62 |
2.8849 | 0.6743 | 8.2566 | 0.0 | 63 |
2.8299 | 0.6809 | 8.3129 | 0.0 | 64 |
2.7877 | 0.6776 | 8.2835 | 0.0 | 65 |
2.7433 | 0.7303 | 8.2969 | 0.0 | 66 |
2.6965 | 0.7072 | 8.3567 | 0.0 | 67 |
2.6248 | 0.7401 | 8.3652 | 0.0 | 68 |
2.6024 | 0.7138 | 8.3640 | 0.0 | 69 |
2.5620 | 0.7303 | 8.3989 | 0.0 | 70 |
2.4997 | 0.7599 | 8.3951 | 0.0 | 71 |
2.4770 | 0.7599 | 8.4202 | 0.0 | 72 |
2.4099 | 0.7697 | 8.4624 | 0.0 | 73 |
2.3756 | 0.7796 | 8.4765 | 0.0 | 74 |
2.3306 | 0.7664 | 8.4952 | 0.0 | 75 |
2.2856 | 0.8158 | 8.5240 | 0.0 | 76 |
2.2484 | 0.7993 | 8.5551 | 0.0 | 77 |
2.2129 | 0.8125 | 8.5684 | 0.0 | 78 |
2.1703 | 0.8059 | 8.5824 | 0.0 | 79 |
2.1305 | 0.8092 | 8.5984 | 0.0 | 80 |
2.1133 | 0.8158 | 8.6333 | 0.0 | 81 |
2.0557 | 0.8026 | 8.6267 | 0.0 | 82 |
2.0275 | 0.8158 | 8.6613 | 0.0 | 83 |
1.9983 | 0.8454 | 8.6840 | 0.0 | 84 |
1.9477 | 0.8322 | 8.6882 | 0.0 | 85 |
1.9220 | 0.8191 | 8.7333 | 0.0 | 86 |
1.8937 | 0.8454 | 8.7709 | 0.0 | 87 |
1.8410 | 0.8388 | 8.7678 | 0.0 | 88 |
1.8068 | 0.8651 | 8.7997 | 0.0 | 89 |
1.7873 | 0.8520 | 8.8332 | 0.0 | 90 |
1.7496 | 0.8684 | 8.8410 | 0.0 | 91 |
1.7219 | 0.875 | 8.8712 | 0.0 | 92 |
1.7160 | 0.8717 | 8.8739 | 0.0 | 93 |
1.6606 | 0.8322 | 8.9188 | 0.0 | 94 |
1.6345 | 0.8618 | 8.9139 | 0.0 | 95 |
1.5970 | 0.8717 | 8.9594 | 0.0 | 96 |
1.5715 | 0.8882 | 8.9689 | 0.0 | 97 |
1.5490 | 0.8783 | 8.9827 | 0.0 | 98 |
1.5315 | 0.8651 | 9.0076 | 0.0 | 99 |
Framework versions
- Transformers 4.19.2
- TensorFlow 2.9.1
- Datasets 2.1.0
- Tokenizers 0.12.1