<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
faiq-wav2vec2-large-xlsr-indo-demo-v100-newparameter
This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the common_voice_11_0 dataset. It achieves the following results on the evaluation set:
- Loss: 0.2880
- Wer: 0.3926
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 300
- num_epochs: 60
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Wer |
---|---|---|---|---|
11.1998 | 0.73 | 100 | 4.4160 | 1.0 |
3.166 | 1.46 | 200 | 2.9133 | 1.0 |
2.89 | 2.19 | 300 | 2.8775 | 1.0 |
2.8188 | 2.92 | 400 | 2.7385 | 1.0007 |
2.656 | 3.65 | 500 | 1.9852 | 1.0208 |
1.5392 | 4.38 | 600 | 0.6224 | 0.7274 |
1.0493 | 5.11 | 700 | 0.4720 | 0.6435 |
0.9097 | 5.84 | 800 | 0.4157 | 0.5892 |
0.8133 | 6.57 | 900 | 0.3720 | 0.5668 |
0.7818 | 7.3 | 1000 | 0.3570 | 0.5416 |
0.7376 | 8.03 | 1100 | 0.3346 | 0.5209 |
0.7131 | 8.76 | 1200 | 0.3234 | 0.5124 |
0.6624 | 9.49 | 1300 | 0.3313 | 0.5005 |
0.6609 | 10.22 | 1400 | 0.3162 | 0.4882 |
0.6331 | 10.95 | 1500 | 0.3257 | 0.4850 |
0.6123 | 11.68 | 1600 | 0.3271 | 0.4804 |
0.6018 | 12.41 | 1700 | 0.3052 | 0.4729 |
0.5822 | 13.14 | 1800 | 0.3048 | 0.4642 |
0.5724 | 13.87 | 1900 | 0.3058 | 0.4698 |
0.5519 | 14.6 | 2000 | 0.2952 | 0.4601 |
0.54 | 15.33 | 2100 | 0.2889 | 0.4582 |
0.5393 | 16.06 | 2200 | 0.2875 | 0.4491 |
0.5273 | 16.79 | 2300 | 0.2843 | 0.4465 |
0.5145 | 17.52 | 2400 | 0.2782 | 0.4403 |
0.515 | 18.25 | 2500 | 0.2916 | 0.4457 |
0.5006 | 18.98 | 2600 | 0.2795 | 0.4418 |
0.5025 | 19.71 | 2700 | 0.2788 | 0.4360 |
0.491 | 20.44 | 2800 | 0.2903 | 0.4348 |
0.478 | 21.17 | 2900 | 0.2819 | 0.4264 |
0.4721 | 21.9 | 3000 | 0.2842 | 0.4258 |
0.4709 | 22.63 | 3100 | 0.2865 | 0.4316 |
0.4618 | 23.36 | 3200 | 0.2911 | 0.4414 |
0.4592 | 24.09 | 3300 | 0.2901 | 0.4267 |
0.4448 | 24.82 | 3400 | 0.2871 | 0.4247 |
0.4347 | 25.55 | 3500 | 0.2772 | 0.4225 |
0.4515 | 26.28 | 3600 | 0.2907 | 0.4203 |
0.4368 | 27.01 | 3700 | 0.2749 | 0.4175 |
0.4317 | 27.74 | 3800 | 0.2781 | 0.4204 |
0.4158 | 28.47 | 3900 | 0.2847 | 0.4214 |
0.4225 | 29.2 | 4000 | 0.2815 | 0.4160 |
0.4205 | 29.93 | 4100 | 0.2792 | 0.4088 |
0.4035 | 30.66 | 4200 | 0.2801 | 0.4097 |
0.405 | 31.39 | 4300 | 0.2853 | 0.4105 |
0.404 | 32.12 | 4400 | 0.2728 | 0.4083 |
0.3929 | 32.85 | 4500 | 0.2801 | 0.4118 |
0.4022 | 33.58 | 4600 | 0.2801 | 0.4058 |
0.3951 | 34.31 | 4700 | 0.2857 | 0.4095 |
0.3716 | 35.04 | 4800 | 0.2882 | 0.4055 |
0.3786 | 35.77 | 4900 | 0.2866 | 0.4056 |
0.3886 | 36.5 | 5000 | 0.2924 | 0.4090 |
0.3746 | 37.23 | 5100 | 0.2820 | 0.4066 |
0.3883 | 37.96 | 5200 | 0.2730 | 0.4006 |
0.374 | 38.69 | 5300 | 0.2825 | 0.3992 |
0.3741 | 39.42 | 5400 | 0.2950 | 0.4011 |
0.3709 | 40.15 | 5500 | 0.2930 | 0.4015 |
0.3579 | 40.88 | 5600 | 0.2919 | 0.4015 |
0.3704 | 41.61 | 5700 | 0.2826 | 0.4014 |
0.3648 | 42.34 | 5800 | 0.2847 | 0.4001 |
0.3549 | 43.07 | 5900 | 0.2934 | 0.4020 |
0.3522 | 43.8 | 6000 | 0.2846 | 0.4004 |
0.3575 | 44.53 | 6100 | 0.2892 | 0.3986 |
0.3512 | 45.26 | 6200 | 0.2952 | 0.4008 |
0.3525 | 45.99 | 6300 | 0.2918 | 0.3961 |
0.3414 | 46.72 | 6400 | 0.2884 | 0.3945 |
0.3458 | 47.45 | 6500 | 0.2918 | 0.3979 |
0.337 | 48.18 | 6600 | 0.2838 | 0.3933 |
0.3352 | 48.91 | 6700 | 0.2872 | 0.3939 |
0.3374 | 49.64 | 6800 | 0.2860 | 0.3941 |
0.327 | 50.36 | 6900 | 0.2820 | 0.3920 |
0.3396 | 51.09 | 7000 | 0.2884 | 0.3946 |
0.3246 | 51.82 | 7100 | 0.2960 | 0.3930 |
0.322 | 52.55 | 7200 | 0.2881 | 0.3949 |
0.331 | 53.28 | 7300 | 0.2927 | 0.3930 |
0.3406 | 54.01 | 7400 | 0.2940 | 0.3954 |
0.3292 | 54.74 | 7500 | 0.2873 | 0.3946 |
0.3209 | 55.47 | 7600 | 0.2881 | 0.3915 |
0.3275 | 56.2 | 7700 | 0.2921 | 0.3922 |
0.3311 | 56.93 | 7800 | 0.2877 | 0.3915 |
0.3282 | 57.66 | 7900 | 0.2866 | 0.3931 |
0.3255 | 58.39 | 8000 | 0.2863 | 0.3917 |
0.3182 | 59.12 | 8100 | 0.2878 | 0.3931 |
0.3321 | 59.85 | 8200 | 0.2880 | 0.3926 |
Framework versions
- Transformers 4.28.0
- Pytorch 2.0.0+cu118
- Datasets 2.6.1
- Tokenizers 0.13.3