<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
wav2vec2-burak-new-300-v2-9-medium
This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.3098
- Wer: 0.1789
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 271
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer |
|---|---|---|---|---|
| 3.2366 | 9.43 | 500 | 0.3980 | 0.4652 |
| 0.5066 | 18.87 | 1000 | 0.2423 | 0.2719 |
| 0.2559 | 28.3 | 1500 | 0.2482 | 0.2443 |
| 0.1869 | 37.74 | 2000 | 0.2537 | 0.2395 |
| 0.1498 | 47.17 | 2500 | 0.2877 | 0.2361 |
| 0.1271 | 56.6 | 3000 | 0.2681 | 0.2237 |
| 0.1145 | 66.04 | 3500 | 0.2788 | 0.2189 |
| 0.1043 | 75.47 | 4000 | 0.2800 | 0.2264 |
| 0.094 | 84.91 | 4500 | 0.2992 | 0.2244 |
| 0.0844 | 94.34 | 5000 | 0.2864 | 0.2209 |
| 0.0776 | 103.77 | 5500 | 0.2758 | 0.2175 |
| 0.0714 | 113.21 | 6000 | 0.2792 | 0.2051 |
| 0.0666 | 122.64 | 6500 | 0.2945 | 0.2175 |
| 0.0601 | 132.08 | 7000 | 0.2865 | 0.2092 |
| 0.0579 | 141.51 | 7500 | 0.3168 | 0.2175 |
| 0.0532 | 150.94 | 8000 | 0.3110 | 0.2292 |
| 0.0474 | 160.38 | 8500 | 0.3070 | 0.2175 |
| 0.0446 | 169.81 | 9000 | 0.3206 | 0.2223 |
| 0.0409 | 179.25 | 9500 | 0.3017 | 0.2106 |
| 0.037 | 188.68 | 10000 | 0.3157 | 0.2092 |
| 0.0344 | 198.11 | 10500 | 0.3222 | 0.2058 |
| 0.0345 | 207.55 | 11000 | 0.3047 | 0.2017 |
| 0.0309 | 216.98 | 11500 | 0.3023 | 0.1913 |
| 0.03 | 226.42 | 12000 | 0.2963 | 0.1920 |
| 0.0268 | 235.85 | 12500 | 0.3036 | 0.1872 |
| 0.0249 | 245.28 | 13000 | 0.2926 | 0.1844 |
| 0.0227 | 254.72 | 13500 | 0.3045 | 0.1865 |
| 0.021 | 264.15 | 14000 | 0.3098 | 0.1789 |
Framework versions
- Transformers 4.25.1
- Pytorch 1.13.0+cu116
- Datasets 2.7.1
- Tokenizers 0.13.2
