<!-- This model card has been generated automatically according to the information Keras had access to. You should probably proofread and complete it, then remove this comment. -->
pretrained-m-bert-1
This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:
- Train Loss: 3.3662
- Validation Loss: 14.3784
- Epoch: 99
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'learning_rate': 1e-04, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32
Training results
| Train Loss | Validation Loss | Epoch |
|---|---|---|
| 10.2573 | 10.9396 | 0 |
| 7.8912 | 10.9597 | 1 |
| 6.8354 | 11.2198 | 2 |
| 6.3659 | 11.5487 | 3 |
| 6.0787 | 11.1321 | 4 |
| 5.8528 | 11.1589 | 5 |
| 5.7689 | 11.3702 | 6 |
| 5.3858 | 11.8196 | 7 |
| 5.1194 | 12.0623 | 8 |
| 5.2013 | 11.8812 | 9 |
| 5.0820 | 11.7143 | 10 |
| 4.9735 | 12.6034 | 11 |
| 5.0704 | 11.9522 | 12 |
| 5.0098 | 12.1079 | 13 |
| 5.0869 | 12.2301 | 14 |
| 4.8156 | 12.2691 | 15 |
| 4.8344 | 12.6859 | 16 |
| 4.8416 | 12.5824 | 17 |
| 4.5567 | 13.1279 | 18 |
| 4.6677 | 12.5301 | 19 |
| 4.7248 | 12.5819 | 20 |
| 4.5435 | 12.5466 | 21 |
| 4.6501 | 12.4965 | 22 |
| 4.7348 | 12.7858 | 23 |
| 4.4649 | 12.5727 | 24 |
| 4.3665 | 12.2411 | 25 |
| 4.6434 | 12.6519 | 26 |
| 4.4239 | 13.6048 | 27 |
| 4.3893 | 13.3377 | 28 |
| 4.6514 | 12.3628 | 29 |
| 4.4743 | 12.6613 | 30 |
| 4.2777 | 12.3419 | 31 |
| 4.5667 | 13.5223 | 32 |
| 4.0886 | 13.2224 | 33 |
| 4.3032 | 13.2787 | 34 |
| 4.3670 | 13.1041 | 35 |
| 4.1487 | 12.6903 | 36 |
| 4.3331 | 13.4880 | 37 |
| 4.2907 | 13.2240 | 38 |
| 4.3252 | 12.6439 | 39 |
| 4.1121 | 13.5487 | 40 |
| 4.3273 | 14.4200 | 41 |
| 4.2030 | 14.4691 | 42 |
| 4.1795 | 13.1436 | 43 |
| 4.0424 | 14.0504 | 44 |
| 4.0158 | 12.5468 | 45 |
| 4.0108 | 13.6426 | 46 |
| 3.9515 | 13.4965 | 47 |
| 3.9743 | 13.2319 | 48 |
| 4.1075 | 13.2999 | 49 |
| 4.0501 | 12.7201 | 50 |
| 3.8606 | 13.1704 | 51 |
| 3.8056 | 13.7504 | 52 |
| 3.7682 | 13.5004 | 53 |
| 4.0676 | 13.6444 | 54 |
| 4.0957 | 13.4160 | 55 |
| 4.9373 | 14.2742 | 56 |
| 4.5111 | 13.9469 | 57 |
| 4.1604 | 13.4773 | 58 |
| 3.9956 | 12.9802 | 59 |
| 4.1232 | 14.1715 | 60 |
| 3.9857 | 12.2465 | 61 |
| 4.1082 | 13.8947 | 62 |
| 3.8659 | 13.6370 | 63 |
| 3.8396 | 13.5898 | 64 |
| 3.8220 | 13.2523 | 65 |
| 3.6864 | 13.9323 | 66 |
| 3.7541 | 13.6081 | 67 |
| 3.8218 | 12.9945 | 68 |
| 3.7251 | 13.7039 | 69 |
| 3.5017 | 12.9811 | 70 |
| 3.5342 | 12.5702 | 71 |
| 3.9520 | 12.4899 | 72 |
| 3.7465 | 13.2309 | 73 |
| 3.6003 | 14.0988 | 74 |
| 3.7954 | 13.0785 | 75 |
| 3.5654 | 13.7277 | 76 |
| 3.5591 | 13.7914 | 77 |
| 3.5355 | 13.6749 | 78 |
| 3.5903 | 13.6141 | 79 |
| 3.5371 | 13.4166 | 80 |
| 3.4502 | 12.6523 | 81 |
| 3.3372 | 13.8609 | 82 |
| 3.3071 | 14.3441 | 83 |
| 3.6932 | 13.9718 | 84 |
| 3.5619 | 13.3749 | 85 |
| 3.5016 | 13.1467 | 86 |
| 3.4279 | 14.0124 | 87 |
| 3.3140 | 13.6681 | 88 |
| 3.3575 | 12.9451 | 89 |
| 3.2268 | 12.4299 | 90 |
| 3.2001 | 14.5106 | 91 |
| 3.1390 | 13.9366 | 92 |
| 3.1230 | 13.6865 | 93 |
| 3.2337 | 13.5835 | 94 |
| 3.1397 | 13.8130 | 95 |
| 3.2095 | 13.8431 | 96 |
| 2.9553 | 14.5159 | 97 |
| 3.3319 | 12.9168 | 98 |
| 3.3662 | 14.3784 | 99 |
Framework versions
- Transformers 4.27.0.dev0
- TensorFlow 2.9.2
- Datasets 2.9.0
- Tokenizers 0.13.2
