<!-- This model card has been generated automatically according to the information Keras had access to. You should probably proofread and complete it, then remove this comment. -->
UrduGPT2
This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:
- Train Loss: 0.5016
- Train Accuracy: 0.9034
- Epoch: 54
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': 1.0, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 3e-05, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
- training_precision: float32
Training results
| Train Loss | Train Accuracy | Epoch |
|---|---|---|
| 6.5740 | 0.1155 | 0 |
| 4.9828 | 0.2007 | 1 |
| 4.6460 | 0.2287 | 2 |
| 4.4430 | 0.2473 | 3 |
| 4.2933 | 0.2617 | 4 |
| 4.1681 | 0.2744 | 5 |
| 4.0549 | 0.2868 | 6 |
| 3.9505 | 0.2989 | 7 |
| 3.8502 | 0.3109 | 8 |
| 3.7497 | 0.3245 | 9 |
| 3.6537 | 0.3372 | 10 |
| 3.5550 | 0.3511 | 11 |
| 3.4564 | 0.3648 | 12 |
| 3.3584 | 0.3794 | 13 |
| 3.2613 | 0.3947 | 14 |
| 3.1628 | 0.4099 | 15 |
| 3.0653 | 0.4259 | 16 |
| 2.9669 | 0.4418 | 17 |
| 2.8693 | 0.4576 | 18 |
| 2.7689 | 0.4754 | 19 |
| 2.6727 | 0.4920 | 20 |
| 2.5747 | 0.5088 | 21 |
| 2.4819 | 0.5245 | 22 |
| 2.3828 | 0.5421 | 23 |
| 2.2905 | 0.5591 | 24 |
| 2.1991 | 0.5750 | 25 |
| 2.1100 | 0.5918 | 26 |
| 2.0228 | 0.6076 | 27 |
| 1.9335 | 0.6238 | 28 |
| 1.8516 | 0.6391 | 29 |
| 1.7704 | 0.6546 | 30 |
| 1.6923 | 0.6688 | 31 |
| 1.6178 | 0.6825 | 32 |
| 1.5407 | 0.6974 | 33 |
| 1.4728 | 0.7100 | 34 |
| 1.4049 | 0.7230 | 35 |
| 1.3378 | 0.7362 | 36 |
| 1.2763 | 0.7478 | 37 |
| 1.2118 | 0.7603 | 38 |
| 1.1526 | 0.7716 | 39 |
| 1.0974 | 0.7814 | 40 |
| 1.0433 | 0.7927 | 41 |
| 0.9909 | 0.8024 | 42 |
| 0.9400 | 0.8128 | 43 |
| 0.8926 | 0.8226 | 44 |
| 0.8440 | 0.8318 | 45 |
| 0.7999 | 0.8413 | 46 |
| 0.7558 | 0.8499 | 47 |
| 0.7160 | 0.8579 | 48 |
| 0.6750 | 0.8673 | 49 |
| 0.6360 | 0.8753 | 50 |
| 0.6015 | 0.8822 | 51 |
| 0.5657 | 0.8897 | 52 |
| 0.5339 | 0.8961 | 53 |
| 0.5016 | 0.9034 | 54 |
Framework versions
- Transformers 4.30.2
- TensorFlow 2.12.0
- Datasets 2.1.0
- Tokenizers 0.13.3
