<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
model
This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.6003
- Wer Ortho: 72.9876
- Wer: 49.5585
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant_with_warmup
- lr_scheduler_warmup_steps: 5
- training_steps: 100
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer | Wer Ortho |
|---|---|---|---|---|---|
| 1.5813 | 0.04 | 5 | 1.2523 | 73.4580 | 94.3667 |
| 0.9924 | 0.08 | 10 | 1.0420 | 76.0991 | 94.0559 |
| 0.8522 | 0.12 | 15 | 0.9581 | 71.8012 | 91.8415 |
| 0.7602 | 0.17 | 20 | 0.9154 | 64.7802 | 88.3450 |
| 0.6813 | 0.21 | 25 | 0.8780 | 65.9285 | 87.9953 |
| 0.7084 | 0.25 | 30 | 0.8520 | 63.5171 | 86.4413 |
| 0.5974 | 0.29 | 35 | 0.8120 | 62.9593 | 85.5866 |
| 0.5631 | 0.33 | 40 | 0.7904 | 59.9409 | 84.1492 |
| 0.6962 | 0.38 | 45 | 0.7631 | 58.0709 | 82.6340 |
| 0.4946 | 0.42 | 50 | 0.7486 | 60.7448 | 83.1779 |
| 0.6118 | 0.46 | 55 | 0.7216 | 59.8097 | 84.4600 |
| 0.5004 | 0.5 | 60 | 0.7018 | 58.8255 | 82.4786 |
| 0.4357 | 0.54 | 65 | 0.6932 | 57.4311 | 81.2743 |
| 0.4478 | 0.58 | 70 | 0.6839 | 55.8399 | 80.8469 |
| 0.4561 | 0.62 | 75 | 0.6856 | 59.3012 | 80.8081 |
| 0.429 | 0.67 | 80 | 0.6646 | 55.0853 | 79.9145 |
| 0.4072 | 0.71 | 85 | 0.6601 | 52.2638 | 76.0684 |
| 0.4096 | 0.75 | 90 | 0.6477 | 52.3950 | 76.8842 |
| 0.4471 | 0.79 | 95 | 0.6393 | 51.2795 | 76.1072 |
| 0.3282 | 0.83 | 100 | 0.6003 | 72.9876 | 49.5585 |
Framework versions
- Transformers 4.34.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1
