<!-- This model card has been generated automatically according to the information Keras had access to. You should probably proofread and complete it, then remove this comment. -->
skillsBERT_v2_tf_epoch100
This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:
- Train Loss: 0.0434
- Validation Loss: 7.6168
- Epoch: 99
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'AdamW', 'weight_decay': 0.004, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': False, 'is_legacy_optimizer': False, 'learning_rate': 5e-05, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32
Training results
| Train Loss | Validation Loss | Epoch |
|---|---|---|
| 6.9352 | 6.9523 | 0 |
| 6.2407 | 6.4961 | 1 |
| 5.3385 | 5.4546 | 2 |
| 4.5980 | 5.2382 | 3 |
| 4.1083 | 4.9088 | 4 |
| 3.7477 | 4.7707 | 5 |
| 3.4448 | 4.7855 | 6 |
| 3.1698 | 4.8487 | 7 |
| 2.9094 | 4.6565 | 8 |
| 2.6538 | 4.7127 | 9 |
| 2.4067 | 4.7005 | 10 |
| 2.1610 | 4.7585 | 11 |
| 1.9259 | 4.9684 | 12 |
| 1.6975 | 5.1386 | 13 |
| 1.4856 | 5.1549 | 14 |
| 1.2886 | 5.1025 | 15 |
| 1.1029 | 5.1759 | 16 |
| 0.9386 | 5.4045 | 17 |
| 0.7907 | 5.6535 | 18 |
| 0.6633 | 5.6907 | 19 |
| 0.5530 | 5.7347 | 20 |
| 0.4623 | 5.7088 | 21 |
| 0.3864 | 5.7850 | 22 |
| 0.3280 | 5.8012 | 23 |
| 0.2797 | 6.1019 | 24 |
| 0.2434 | 6.1027 | 25 |
| 0.2097 | 6.3630 | 26 |
| 0.1895 | 6.2085 | 27 |
| 0.1734 | 6.2182 | 28 |
| 0.1578 | 6.3494 | 29 |
| 0.1444 | 6.3452 | 30 |
| 0.1354 | 6.7442 | 31 |
| 0.1313 | 6.5799 | 32 |
| 0.1172 | 6.7019 | 33 |
| 0.1178 | 6.5615 | 34 |
| 0.1115 | 6.6848 | 35 |
| 0.1070 | 6.7984 | 36 |
| 0.1000 | 6.8200 | 37 |
| 0.0998 | 6.9078 | 38 |
| 0.0922 | 6.8531 | 39 |
| 0.0890 | 7.0363 | 40 |
| 0.0880 | 6.9250 | 41 |
| 0.0852 | 7.1117 | 42 |
| 0.0841 | 6.8522 | 43 |
| 0.0818 | 6.9779 | 44 |
| 0.0785 | 7.0038 | 45 |
| 0.0779 | 6.9143 | 46 |
| 0.0731 | 7.0954 | 47 |
| 0.0734 | 7.3798 | 48 |
| 0.0706 | 7.2156 | 49 |
| 0.0713 | 7.0388 | 50 |
| 0.0699 | 7.0339 | 51 |
| 0.0667 | 7.1569 | 52 |
| 0.0664 | 7.3617 | 53 |
| 0.0647 | 7.2491 | 54 |
| 0.0647 | 7.2064 | 55 |
| 0.0658 | 7.3093 | 56 |
| 0.0612 | 7.2296 | 57 |
| 0.0602 | 7.2176 | 58 |
| 0.0613 | 7.3468 | 59 |
| 0.0587 | 7.3203 | 60 |
| 0.0584 | 7.5229 | 61 |
| 0.0571 | 7.4048 | 62 |
| 0.0585 | 7.2250 | 63 |
| 0.0575 | 7.1258 | 64 |
| 0.0548 | 7.3677 | 65 |
| 0.0556 | 7.6926 | 66 |
| 0.0545 | 7.3272 | 67 |
| 0.0542 | 7.5853 | 68 |
| 0.0545 | 7.5508 | 69 |
| 0.0525 | 7.4420 | 70 |
| 0.0526 | 7.3759 | 71 |
| 0.0515 | 7.3879 | 72 |
| 0.0502 | 7.4672 | 73 |
| 0.0532 | 7.2759 | 74 |
| 0.0493 | 7.6825 | 75 |
| 0.0469 | 7.8799 | 76 |
| 0.0463 | 7.5466 | 77 |
| 0.0480 | 7.5932 | 78 |
| 0.0505 | 7.5632 | 79 |
| 0.0477 | 7.7247 | 80 |
| 0.0474 | 7.4508 | 81 |
| 0.0464 | 7.5723 | 82 |
| 0.0461 | 7.5043 | 83 |
| 0.0470 | 7.6296 | 84 |
| 0.0455 | 7.6214 | 85 |
| 0.0479 | 7.4556 | 86 |
| 0.0444 | 7.3914 | 87 |
| 0.0445 | 7.6954 | 88 |
| 0.0457 | 7.6601 | 89 |
| 0.0453 | 7.4933 | 90 |
| 0.0443 | 7.7381 | 91 |
| 0.0429 | 7.6079 | 92 |
| 0.0413 | 7.7611 | 93 |
| 0.0451 | 7.8315 | 94 |
| 0.0397 | 7.7161 | 95 |
| 0.0450 | 7.9785 | 96 |
| 0.0386 | 7.7460 | 97 |
| 0.0434 | 7.7243 | 98 |
| 0.0434 | 7.6168 | 99 |
Framework versions
- Transformers 4.28.0.dev0
- TensorFlow 2.12.0
- Datasets 2.11.0
- Tokenizers 0.13.2
