<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
dgx1_distil_w2v2_base_mozilla_12_to_6_batch_16_epoch_50_try2
This model is a fine-tuned version of rohitp1/dgx1_distil_w2v2_base_mozilla_12_to_6_batch_16_epoch_20 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 41.1845
- Wer: 0.3475
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0005
- train_batch_size: 16
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 128
- total_train_batch_size: 2048
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.2
- num_epochs: 50
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer |
|---|---|---|---|---|
| 325.2839 | 2.2 | 150 | 27.1160 | 0.3780 |
| 316.7718 | 4.41 | 300 | 27.4142 | 0.3772 |
| 299.5312 | 6.61 | 450 | 28.4325 | 0.3764 |
| 286.0595 | 8.82 | 600 | 28.6401 | 0.3746 |
| 274.714 | 11.03 | 750 | 28.8407 | 0.3752 |
| 257.4291 | 13.23 | 900 | 30.4966 | 0.3708 |
| 246.298 | 15.44 | 1050 | 32.1811 | 0.3665 |
| 231.0542 | 17.64 | 1200 | 32.0769 | 0.3721 |
| 217.6621 | 19.85 | 1350 | 33.4042 | 0.3621 |
| 209.504 | 22.06 | 1500 | 31.9015 | 0.3636 |
| 198.3204 | 24.26 | 1650 | 34.1675 | 0.3606 |
| 188.921 | 26.47 | 1800 | 35.2466 | 0.3625 |
| 181.8529 | 28.67 | 1950 | 34.7835 | 0.3574 |
| 174.2548 | 30.88 | 2100 | 36.6986 | 0.3605 |
| 166.8866 | 33.09 | 2250 | 38.0683 | 0.3584 |
| 160.1716 | 35.29 | 2400 | 35.6336 | 0.3552 |
| 154.5262 | 37.5 | 2550 | 39.7696 | 0.3541 |
| 150.0273 | 39.7 | 2700 | 39.7674 | 0.3542 |
| 149.0675 | 41.91 | 2850 | 41.1694 | 0.3532 |
| 142.0086 | 44.12 | 3000 | 41.1995 | 0.3593 |
| 139.7779 | 46.32 | 3150 | 40.9558 | 0.3459 |
| 133.4296 | 48.53 | 3300 | 41.1845 | 0.3475 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.12.1
- Datasets 2.8.0
- Tokenizers 0.13.2
