<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
base-model-with-warmup-fulldata-LR-3e4-fairseq-V1
This model is a fine-tuned version of facebook/wav2vec2-base-960h on the None dataset. It achieves the following results on the evaluation set:
- Loss: 4.6024
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 50
Training results
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
12.0385 | 2.05 | 200 | 4.6950 |
3.9271 | 4.1 | 400 | 4.2602 |
3.8802 | 6.15 | 600 | 4.2946 |
3.9082 | 8.21 | 800 | 4.6729 |
3.8298 | 10.26 | 1000 | 5.2657 |
3.8352 | 12.31 | 1200 | 5.0217 |
3.8661 | 14.36 | 1400 | 4.6284 |
3.8028 | 16.41 | 1600 | 4.6804 |
3.8147 | 18.46 | 1800 | 4.6496 |
3.8209 | 20.51 | 2000 | 4.7289 |
3.8015 | 22.56 | 2200 | 4.7908 |
3.8048 | 24.62 | 2400 | 4.4793 |
3.7978 | 26.67 | 2600 | 4.4383 |
3.8001 | 28.72 | 2800 | 4.5666 |
3.795 | 30.77 | 3000 | 4.7433 |
3.7983 | 32.82 | 3200 | 4.5482 |
3.797 | 34.87 | 3400 | 4.5401 |
3.7963 | 36.92 | 3600 | 4.5661 |
3.7927 | 38.97 | 3800 | 4.6994 |
3.7938 | 41.03 | 4000 | 4.5958 |
4.1155 | 43.08 | 4200 | 4.6279 |
3.7862 | 45.13 | 4400 | 4.6126 |
3.7934 | 47.18 | 4600 | 4.5489 |
3.7851 | 49.23 | 4800 | 4.6024 |
Framework versions
- Transformers 4.28.0
- Pytorch 2.0.1
- Datasets 2.12.0
- Tokenizers 0.13.2