<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
timit-distil-kl-alpha-0.25-T-1-take-2
This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:
- Loss: 170.1500
- Wer: 0.7698
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 28
- eval_batch_size: 14
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 56
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.2
- num_epochs: 50
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer |
|---|---|---|---|---|
| 583.5526 | 1.22 | 100 | 354.4852 | 1.0483 |
| 367.4895 | 2.44 | 200 | 246.2063 | 0.9295 |
| 275.3467 | 3.65 | 300 | 216.6919 | 0.8582 |
| 235.4134 | 4.87 | 400 | 200.3912 | 0.8223 |
| 217.7628 | 6.1 | 500 | 191.3873 | 0.8050 |
| 196.5266 | 7.32 | 600 | 186.2596 | 0.7891 |
| 182.3409 | 8.53 | 700 | 180.5465 | 0.7824 |
| 173.6425 | 9.75 | 800 | 180.9611 | 0.7786 |
| 158.4616 | 10.97 | 900 | 174.4431 | 0.7700 |
| 162.9191 | 12.19 | 1000 | 174.9140 | 0.7767 |
| 152.7855 | 13.41 | 1100 | 175.0573 | 0.7697 |
| 146.0713 | 14.63 | 1200 | 174.0519 | 0.7735 |
| 142.7635 | 15.85 | 1300 | 171.1087 | 0.7729 |
| 143.8564 | 17.07 | 1400 | 172.2125 | 0.7665 |
| 138.0579 | 18.29 | 1500 | 171.2589 | 0.7690 |
| 133.162 | 19.51 | 1600 | 169.6842 | 0.7716 |
| 132.8703 | 20.73 | 1700 | 173.7567 | 0.7750 |
| 132.5092 | 21.95 | 1800 | 171.5918 | 0.7658 |
| 133.7408 | 23.17 | 1900 | 170.9486 | 0.7692 |
| 130.2913 | 24.39 | 2000 | 170.2246 | 0.7666 |
| 127.7704 | 25.61 | 2100 | 169.7522 | 0.7680 |
| 126.3399 | 26.82 | 2200 | 171.0318 | 0.7682 |
| 127.5717 | 28.05 | 2300 | 170.2780 | 0.7665 |
| 125.7467 | 29.27 | 2400 | 170.7915 | 0.7689 |
| 119.2796 | 30.48 | 2500 | 170.7032 | 0.7691 |
| 122.8742 | 31.7 | 2600 | 170.5696 | 0.7737 |
| 121.9309 | 32.92 | 2700 | 170.1012 | 0.7721 |
| 122.2507 | 34.15 | 2800 | 170.2254 | 0.7645 |
| 120.9862 | 35.36 | 2900 | 170.4729 | 0.7752 |
| 121.0826 | 36.58 | 3000 | 170.5613 | 0.7747 |
| 119.0979 | 37.8 | 3100 | 170.4102 | 0.7695 |
| 118.2004 | 39.02 | 3200 | 169.9209 | 0.7642 |
| 116.4097 | 40.24 | 3300 | 170.4418 | 0.7685 |
| 117.4168 | 41.46 | 3400 | 171.0443 | 0.7705 |
| 118.362 | 42.68 | 3500 | 169.6040 | 0.7692 |
| 117.1554 | 43.9 | 3600 | 169.7565 | 0.7682 |
| 118.7433 | 45.12 | 3700 | 169.9207 | 0.7719 |
| 115.2184 | 46.34 | 3800 | 170.1183 | 0.7711 |
| 115.6529 | 47.56 | 3900 | 170.0537 | 0.7689 |
| 115.4671 | 48.78 | 4000 | 170.2999 | 0.7700 |
| 114.1326 | 49.99 | 4100 | 170.1500 | 0.7698 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.12.1
- Datasets 2.8.0
- Tokenizers 0.13.2
