<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
wav2vec2-tiny-fp16-demo
This model is a fine-tuned version of yenpolin/wav2vec2-tiny on the DNA_R9.4.1 - NA dataset. It achieves the following results on the evaluation set:
- Loss: 0.5995
- Mean Acc: 51.7421
- Median Acc: 56.8345
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 400
- eval_batch_size: 800
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- num_epochs: 100.0
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Mean Acc | Median Acc |
|---|---|---|---|---|---|
| 1.1754 | 1.0 | 1250 | 1.0956 | 1.8729 | 0.0 |
| 1.0286 | 2.0 | 2500 | 0.9805 | 12.8456 | 0.0 |
| 0.921 | 3.0 | 3750 | 0.9232 | 12.3399 | 0.0 |
| 0.8819 | 4.0 | 5000 | 0.8857 | 18.0811 | 0.0 |
| 0.8574 | 5.0 | 6250 | 0.8691 | 17.4596 | 0.0 |
| 0.8419 | 6.0 | 7500 | 0.8578 | 25.5205 | 0.0 |
| 0.8291 | 7.0 | 8750 | 0.8441 | 23.7467 | 0.0 |
| 0.8167 | 8.0 | 10000 | 0.8346 | 22.1622 | 0.0 |
| 0.801 | 9.0 | 11250 | 0.8108 | 31.3152 | 53.4606 |
| 0.7768 | 10.0 | 12500 | 0.7868 | 30.8526 | 53.2143 |
| 0.7533 | 11.0 | 13750 | 0.7642 | 34.3817 | 53.9589 |
| 0.7385 | 12.0 | 15000 | 0.7637 | 39.9523 | 54.6539 |
| 0.7273 | 13.0 | 16250 | 0.7412 | 38.2095 | 54.5455 |
| 0.7103 | 14.0 | 17500 | 0.7212 | 39.3217 | 54.6032 |
| 0.6876 | 15.0 | 18750 | 0.7128 | 33.8479 | 53.8226 |
| 0.6744 | 16.0 | 20000 | 0.6916 | 36.8302 | 54.0541 |
| 0.6664 | 17.0 | 21250 | 0.6825 | 39.4562 | 54.4928 |
| 0.6598 | 18.0 | 22500 | 0.6753 | 36.5260 | 53.9130 |
| 0.654 | 19.0 | 23750 | 0.6748 | 33.4539 | 53.6058 |
| 0.6492 | 20.0 | 25000 | 0.6702 | 37.7485 | 54.1209 |
| 0.6457 | 21.0 | 26250 | 0.6694 | 40.0633 | 54.4343 |
| 0.6413 | 22.0 | 27500 | 0.6609 | 36.9515 | 53.9062 |
| 0.6377 | 23.0 | 28750 | 0.6576 | 39.6921 | 54.2857 |
| 0.6341 | 24.0 | 30000 | 0.6527 | 41.3149 | 54.5455 |
| 0.6312 | 25.0 | 31250 | 0.6516 | 39.5467 | 54.2587 |
| 0.6279 | 26.0 | 32500 | 0.6487 | 38.4877 | 54.0541 |
| 0.6246 | 27.0 | 33750 | 0.6444 | 43.1149 | 54.7619 |
| 0.6219 | 28.0 | 35000 | 0.6439 | 40.7786 | 54.5151 |
| 0.6191 | 29.0 | 36250 | 0.6418 | 41.0059 | 54.5161 |
| 0.6167 | 30.0 | 37500 | 0.6359 | 42.2812 | 54.7425 |
| 0.6136 | 31.0 | 38750 | 0.6346 | 41.2515 | 54.5977 |
| 0.6111 | 32.0 | 40000 | 0.6332 | 41.7222 | 54.6667 |
| 0.6087 | 33.0 | 41250 | 0.6333 | 41.3847 | 54.5455 |
| 0.6062 | 34.0 | 42500 | 0.6279 | 43.7450 | 55.0000 |
| 0.6042 | 35.0 | 43750 | 0.6269 | 42.5222 | 54.7619 |
| 0.6021 | 36.0 | 45000 | 0.6288 | 43.8255 | 54.9133 |
| 0.6004 | 37.0 | 46250 | 0.6292 | 43.5378 | 55.0186 |
| 0.5987 | 38.0 | 47500 | 0.6233 | 43.8364 | 55.0000 |
| 0.5969 | 39.0 | 48750 | 0.6213 | 44.3750 | 55.1155 |
| 0.595 | 40.0 | 50000 | 0.6207 | 40.1240 | 54.5732 |
| 0.5934 | 41.0 | 51250 | 0.6198 | 44.3429 | 55.0885 |
| 0.5914 | 42.0 | 52500 | 0.6178 | 44.0396 | 55.1181 |
| 0.5895 | 43.0 | 53750 | 0.6155 | 45.7166 | 55.4017 |
| 0.5874 | 44.0 | 55000 | 0.6159 | 46.3816 | 55.4745 |
| 0.5853 | 45.0 | 56250 | 0.6136 | 45.7973 | 55.4810 |
| 0.5834 | 46.0 | 57500 | 0.6142 | 44.1374 | 55.2 |
| 0.5819 | 47.0 | 58750 | 0.6204 | 46.3006 | 55.4945 |
| 0.5802 | 48.0 | 60000 | 0.6093 | 47.6637 | 55.7276 |
| 0.5785 | 49.0 | 61250 | 0.6098 | 46.7728 | 55.625 |
| 0.5771 | 50.0 | 62500 | 0.6075 | 47.4424 | 55.7423 |
| 0.5751 | 51.0 | 63750 | 0.6066 | 46.8072 | 55.7377 |
| 0.5734 | 52.0 | 65000 | 0.6053 | 48.7600 | 56.0109 |
| 0.5719 | 53.0 | 66250 | 0.6048 | 48.8645 | 55.9557 |
| 0.5705 | 54.0 | 67500 | 0.6030 | 48.4536 | 55.9557 |
| 0.5693 | 55.0 | 68750 | 0.6046 | 49.0295 | 56.0870 |
| 0.5677 | 56.0 | 70000 | 0.6034 | 47.9864 | 55.9767 |
| 0.5665 | 57.0 | 71250 | 0.6033 | 48.4370 | 56.0241 |
| 0.5653 | 58.0 | 72500 | 0.6016 | 49.2084 | 56.1069 |
| 0.5635 | 59.0 | 73750 | 0.6021 | 49.6844 | 56.2648 |
| 0.5621 | 60.0 | 75000 | 0.6001 | 50.2077 | 56.3218 |
| 0.5607 | 61.0 | 76250 | 0.6007 | 49.1341 | 56.1753 |
| 0.5592 | 62.0 | 77500 | 0.6011 | 48.9626 | 56.2016 |
| 0.5582 | 63.0 | 78750 | 0.6008 | 49.6105 | 56.2982 |
| 0.557 | 64.0 | 80000 | 0.6012 | 49.5089 | 56.2814 |
| 0.5557 | 65.0 | 81250 | 0.5994 | 49.9531 | 56.3246 |
| 0.5542 | 66.0 | 82500 | 0.6003 | 50.2286 | 56.4706 |
| 0.5533 | 67.0 | 83750 | 0.5990 | 49.9327 | 56.3725 |
| 0.5521 | 68.0 | 85000 | 0.5995 | 50.1928 | 56.4516 |
| 0.5511 | 69.0 | 86250 | 0.5979 | 50.2263 | 56.4444 |
| 0.55 | 70.0 | 87500 | 0.6005 | 50.4875 | 56.5341 |
| 0.5484 | 71.0 | 88750 | 0.5984 | 50.0615 | 56.4626 |
| 0.5475 | 72.0 | 90000 | 0.5991 | 50.4546 | 56.5598 |
| 0.5467 | 73.0 | 91250 | 0.5972 | 50.8276 | 56.5728 |
| 0.5458 | 74.0 | 92500 | 0.5987 | 51.1359 | 56.5934 |
| 0.5445 | 75.0 | 93750 | 0.5986 | 51.0738 | 56.6474 |
| 0.5439 | 76.0 | 95000 | 0.5991 | 50.7110 | 56.6102 |
| 0.5431 | 77.0 | 96250 | 0.5986 | 51.0240 | 56.6667 |
| 0.5423 | 78.0 | 97500 | 0.5992 | 51.0228 | 56.6901 |
| 0.5413 | 79.0 | 98750 | 0.5992 | 50.8097 | 56.6343 |
| 0.5407 | 80.0 | 100000 | 0.5993 | 51.2619 | 56.7251 |
| 0.5399 | 81.0 | 101250 | 0.5975 | 51.4024 | 56.7282 |
| 0.5393 | 82.0 | 102500 | 0.5982 | 51.2136 | 56.7358 |
| 0.5386 | 83.0 | 103750 | 0.5983 | 51.4384 | 56.7376 |
| 0.538 | 84.0 | 105000 | 0.5988 | 51.6233 | 56.7568 |
| 0.5374 | 85.0 | 106250 | 0.5983 | 51.4155 | 56.7867 |
| 0.5369 | 86.0 | 107500 | 0.5987 | 51.4146 | 56.7568 |
| 0.5365 | 87.0 | 108750 | 0.5995 | 51.4874 | 56.8 |
| 0.5361 | 88.0 | 110000 | 0.5994 | 51.5007 | 56.7775 |
| 0.5357 | 89.0 | 111250 | 0.5991 | 51.5986 | 56.7901 |
| 0.5353 | 90.0 | 112500 | 0.5990 | 51.4792 | 56.7742 |
| 0.535 | 91.0 | 113750 | 0.5993 | 51.8824 | 56.8528 |
| 0.5348 | 92.0 | 115000 | 0.5993 | 51.6289 | 56.8116 |
| 0.5346 | 93.0 | 116250 | 0.5993 | 51.6716 | 56.8254 |
| 0.5344 | 94.0 | 117500 | 0.5991 | 51.5984 | 56.8 |
| 0.5342 | 95.0 | 118750 | 0.5995 | 51.8793 | 56.8245 |
| 0.5341 | 96.0 | 120000 | 0.5994 | 51.7306 | 56.8254 |
| 0.534 | 97.0 | 121250 | 0.5994 | 51.7991 | 56.8306 |
| 0.5339 | 98.0 | 122500 | 0.5993 | 51.8468 | 56.8233 |
| 0.5339 | 99.0 | 123750 | 0.5994 | 51.8060 | 56.8389 |
| 0.5338 | 100.0 | 125000 | 0.5995 | 51.7421 | 56.8345 |
Framework versions
- Transformers 4.26.0
- Pytorch 1.12.1
- Datasets 2.9.0
- Tokenizers 0.13.2
