<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-mse-take-4-unfreeze-extractor
This model is a fine-tuned version of rohitp1/libri-alpha-0.75-Temp-1-attention-3-layers-distil-with-6-layers-mse on the None dataset. It achieves the following results on the evaluation set:
- Loss: 35.4977
- Wer: 0.2414
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 8
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.2
- num_epochs: 40
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Wer |
|---|---|---|---|---|
| 557.7442 | 0.45 | 400 | 28.5437 | 0.3344 |
| 576.4579 | 0.9 | 800 | 28.0582 | 0.3313 |
| 557.924 | 1.35 | 1200 | 28.2032 | 0.3285 |
| 558.8386 | 1.79 | 1600 | 28.0733 | 0.3327 |
| 583.0312 | 2.24 | 2000 | 28.3506 | 0.3254 |
| 559.6182 | 2.69 | 2400 | 27.7517 | 0.3245 |
| 555.811 | 3.14 | 2800 | 28.1994 | 0.3275 |
| 555.9074 | 3.59 | 3200 | 28.2289 | 0.3267 |
| 569.4283 | 4.04 | 3600 | 27.9987 | 0.3247 |
| 523.5996 | 4.48 | 4000 | 27.9328 | 0.3178 |
| 543.8255 | 4.93 | 4400 | 28.0181 | 0.3192 |
| 508.707 | 5.38 | 4800 | 27.8667 | 0.3172 |
| 518.0536 | 5.83 | 5200 | 28.0461 | 0.3120 |
| 516.7025 | 6.28 | 5600 | 28.6324 | 0.3193 |
| 509.9804 | 6.73 | 6000 | 28.8554 | 0.3202 |
| 522.2005 | 7.17 | 6400 | 28.4986 | 0.3173 |
| 501.0925 | 7.62 | 6800 | 28.5744 | 0.3095 |
| 506.2044 | 8.07 | 7200 | 29.1753 | 0.3108 |
| 464.1213 | 8.52 | 7600 | 28.5564 | 0.3080 |
| 483.3067 | 8.97 | 8000 | 28.3099 | 0.3063 |
| 463.7952 | 9.42 | 8400 | 28.4788 | 0.2990 |
| 474.824 | 9.87 | 8800 | 27.5007 | 0.2959 |
| 441.7981 | 10.31 | 9200 | 28.3279 | 0.2906 |
| 445.6532 | 10.76 | 9600 | 27.6901 | 0.2881 |
| 427.3226 | 11.21 | 10000 | 28.5749 | 0.2860 |
| 419.5903 | 11.66 | 10400 | 27.3023 | 0.2825 |
| 425.3329 | 12.11 | 10800 | 28.3225 | 0.2803 |
| 401.3551 | 12.56 | 11200 | 28.1836 | 0.2814 |
| 409.8571 | 13.0 | 11600 | 27.9721 | 0.2806 |
| 382.0269 | 13.45 | 12000 | 28.2285 | 0.2798 |
| 363.1065 | 13.9 | 12400 | 28.9252 | 0.2821 |
| 386.975 | 14.35 | 12800 | 28.7444 | 0.2778 |
| 370.1886 | 14.8 | 13200 | 28.3816 | 0.2738 |
| 385.9398 | 15.25 | 13600 | 29.5411 | 0.2759 |
| 347.4368 | 15.7 | 14000 | 28.5876 | 0.2710 |
| 338.2872 | 16.14 | 14400 | 28.9052 | 0.2709 |
| 347.3471 | 16.59 | 14800 | 28.3766 | 0.2679 |
| 344.1634 | 17.04 | 15200 | 29.3270 | 0.2669 |
| 333.9699 | 17.49 | 15600 | 29.2184 | 0.2656 |
| 326.7914 | 17.94 | 16000 | 29.4644 | 0.2659 |
| 328.6156 | 18.39 | 16400 | 30.1155 | 0.2686 |
| 314.8902 | 18.83 | 16800 | 29.8135 | 0.2653 |
| 320.2311 | 19.28 | 17200 | 30.4169 | 0.2654 |
| 311.5116 | 19.73 | 17600 | 30.7323 | 0.2654 |
| 320.7442 | 20.18 | 18000 | 30.3148 | 0.2616 |
| 310.1395 | 20.63 | 18400 | 30.3432 | 0.2626 |
| 298.6844 | 21.08 | 18800 | 30.3217 | 0.2611 |
| 294.7287 | 21.52 | 19200 | 30.4799 | 0.2574 |
| 301.9398 | 21.97 | 19600 | 29.9043 | 0.2562 |
| 285.6117 | 22.42 | 20000 | 30.6270 | 0.2574 |
| 299.511 | 22.87 | 20400 | 30.4342 | 0.2580 |
| 271.373 | 23.32 | 20800 | 31.1784 | 0.2583 |
| 289.4111 | 23.77 | 21200 | 30.8436 | 0.2562 |
| 266.0083 | 24.22 | 21600 | 31.6785 | 0.2576 |
| 271.6104 | 24.66 | 22000 | 31.7733 | 0.2565 |
| 280.7621 | 25.11 | 22400 | 32.7097 | 0.2564 |
| 254.1648 | 25.56 | 22800 | 33.1091 | 0.2564 |
| 276.6574 | 26.01 | 23200 | 31.9279 | 0.2539 |
| 277.4295 | 26.46 | 23600 | 32.4169 | 0.2522 |
| 268.0675 | 26.91 | 24000 | 32.5259 | 0.2510 |
| 249.2665 | 27.35 | 24400 | 32.4788 | 0.2508 |
| 277.0122 | 27.8 | 24800 | 32.7013 | 0.2517 |
| 250.1679 | 28.25 | 25200 | 32.4869 | 0.2524 |
| 242.7224 | 28.7 | 25600 | 32.2633 | 0.2521 |
| 250.325 | 29.15 | 26000 | 33.0046 | 0.2491 |
| 233.9489 | 29.6 | 26400 | 32.7155 | 0.2485 |
| 246.6027 | 30.04 | 26800 | 33.6882 | 0.2485 |
| 244.4221 | 30.49 | 27200 | 34.2592 | 0.2492 |
| 239.4369 | 30.94 | 27600 | 33.6288 | 0.2492 |
| 239.1851 | 31.39 | 28000 | 34.0746 | 0.2484 |
| 234.8415 | 31.84 | 28400 | 34.1040 | 0.2466 |
| 225.2858 | 32.29 | 28800 | 34.6926 | 0.2483 |
| 241.6866 | 32.74 | 29200 | 34.0598 | 0.2474 |
| 224.4263 | 33.18 | 29600 | 34.8568 | 0.2459 |
| 227.2052 | 33.63 | 30000 | 34.8061 | 0.2456 |
| 226.6837 | 34.08 | 30400 | 34.9184 | 0.2450 |
| 219.9877 | 34.53 | 30800 | 34.8988 | 0.2441 |
| 225.5292 | 34.98 | 31200 | 34.9351 | 0.2447 |
| 215.8455 | 35.43 | 31600 | 34.9351 | 0.2437 |
| 210.303 | 35.87 | 32000 | 35.0217 | 0.2439 |
| 230.9594 | 36.32 | 32400 | 35.4323 | 0.2449 |
| 207.6091 | 36.77 | 32800 | 35.1739 | 0.2439 |
| 202.487 | 37.22 | 33200 | 35.3531 | 0.2441 |
| 209.1144 | 37.67 | 33600 | 35.4137 | 0.2419 |
| 212.8689 | 38.12 | 34000 | 35.4311 | 0.2434 |
| 201.1868 | 38.57 | 34400 | 35.6746 | 0.2426 |
| 206.6466 | 39.01 | 34800 | 35.5530 | 0.2420 |
| 218.2249 | 39.46 | 35200 | 35.4107 | 0.2415 |
| 226.1933 | 39.91 | 35600 | 35.4977 | 0.2414 |
Framework versions
- Transformers 4.24.0
- Pytorch 1.12.1
- Datasets 2.7.1
- Tokenizers 0.11.0
