model_phoneme_onSet1

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.0873
0 Precision: 1.0
0 Recall: 0.9677
0 F1-score: 0.9836
0 Support: 31
1 Precision: 0.9355
1 Recall: 0.9667
1 F1-score: 0.9508
1 Support: 30
2 Precision: 0.9565
2 Recall: 1.0
2 F1-score: 0.9778
2 Support: 22
3 Precision: 1.0
3 Recall: 0.9333
3 F1-score: 0.9655
3 Support: 15
Accuracy: 0.9694
Macro avg Precision: 0.9730
Macro avg Recall: 0.9669
Macro avg F1-score: 0.9694
Macro avg Support: 98
Weighted avg Precision: 0.9705
Weighted avg Recall: 0.9694
Weighted avg F1-score: 0.9695
Weighted avg Support: 98
Wer: 0.0999
Mtrix: [[0, 1, 2, 3], [0, 30, 1, 0, 0], [1, 0, 29, 1, 0], [2, 0, 0, 22, 0], [3, 0, 1, 0, 14]]

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 200
num_epochs: 70
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	0 Precision	0 Recall	0 F1-score	0 Support	1 Precision	1 Recall	1 F1-score	1 Support	2 Precision	2 Recall	2 F1-score	2 Support	3 Precision	3 Recall	3 F1-score	3 Support	Accuracy	Macro avg Precision	Macro avg Recall	Macro avg F1-score	Macro avg Support	Weighted avg Precision	Weighted avg Recall	Weighted avg F1-score	Weighted avg Support	Wer	Mtrix
4.2188	4.16	100	3.4689	1.0	0.2581	0.4103	31	0.0	0.0	0.0	30	0.0	0.0	0.0	22	0.1667	1.0	0.2857	15	0.2347	0.2917	0.3145	0.1740	98	0.3418	0.2347	0.1735	98	0.9980	[[0, 1, 2, 3], [0, 8, 0, 0, 23], [1, 0, 0, 0, 30], [2, 0, 0, 0, 22], [3, 0, 0, 0, 15]]
3.3407	8.33	200	3.1569	1.0	0.2581	0.4103	31	0.0	0.0	0.0	30	0.0	0.0	0.0	22	0.1667	1.0	0.2857	15	0.2347	0.2917	0.3145	0.1740	98	0.3418	0.2347	0.1735	98	0.9980	[[0, 1, 2, 3], [0, 8, 0, 0, 23], [1, 0, 0, 0, 30], [2, 0, 0, 0, 22], [3, 0, 0, 0, 15]]
3.1051	12.49	300	3.1500	1.0	0.2581	0.4103	31	0.0	0.0	0.0	30	0.0	0.0	0.0	22	0.1667	1.0	0.2857	15	0.2347	0.2917	0.3145	0.1740	98	0.3418	0.2347	0.1735	98	0.9980	[[0, 1, 2, 3], [0, 8, 0, 0, 23], [1, 0, 0, 0, 30], [2, 0, 0, 0, 22], [3, 0, 0, 0, 15]]
2.8593	16.65	400	2.7590	0.6889	1.0	0.8158	31	0.0	0.0	0.0	30	0.3962	0.9545	0.5600	22	0.0	0.0	0.0	15	0.5306	0.2713	0.4886	0.3439	98	0.3069	0.5306	0.3838	98	1.0	[[0, 1, 2, 3], [0, 31, 0, 0, 0], [1, 13, 0, 17, 0], [2, 1, 0, 21, 0], [3, 0, 0, 15, 0]]
2.2351	20.82	500	2.0930	0.9118	1.0	0.9538	31	1.0	0.5333	0.6957	30	0.6286	1.0	0.7719	22	0.8462	0.7333	0.7857	15	0.8163	0.8466	0.8167	0.8018	98	0.8652	0.8163	0.8082	98	0.9631	[[0, 1, 2, 3], [0, 31, 0, 0, 0], [1, 3, 16, 9, 2], [2, 0, 0, 22, 0], [3, 0, 0, 4, 11]]
1.8803	24.98	600	1.7480	1.0	1.0	1.0	31	0.9375	1.0	0.9677	30	1.0	0.9545	0.9767	22	1.0	0.9333	0.9655	15	0.9796	0.9844	0.9720	0.9775	98	0.9809	0.9796	0.9796	98	0.9552	[[0, 1, 2, 3], [0, 31, 0, 0, 0], [1, 0, 30, 0, 0], [2, 0, 1, 21, 0], [3, 0, 1, 0, 14]]
1.5034	29.16	700	1.3694	1.0	1.0	1.0	31	1.0	1.0	1.0	30	1.0	1.0	1.0	22	1.0	1.0	1.0	15	1.0	1.0	1.0	1.0	98	1.0	1.0	1.0	98	0.9429	[[0, 1, 2, 3], [0, 31, 0, 0, 0], [1, 0, 30, 0, 0], [2, 0, 0, 22, 0], [3, 0, 0, 0, 15]]
1.0229	33.33	800	0.8522	1.0	1.0	1.0	31	1.0	0.9667	0.9831	30	0.9565	1.0	0.9778	22	1.0	1.0	1.0	15	0.9898	0.9891	0.9917	0.9902	98	0.9902	0.9898	0.9898	98	0.8848	[[0, 1, 2, 3], [0, 31, 0, 0, 0], [1, 0, 29, 1, 0], [2, 0, 0, 22, 0], [3, 0, 0, 0, 15]]
0.4811	37.49	900	0.3999	1.0	1.0	1.0	31	0.9677	1.0	0.9836	30	1.0	1.0	1.0	22	1.0	0.9333	0.9655	15	0.9898	0.9919	0.9833	0.9873	98	0.9901	0.9898	0.9897	98	0.5576	[[0, 1, 2, 3], [0, 31, 0, 0, 0], [1, 0, 30, 0, 0], [2, 0, 0, 22, 0], [3, 0, 1, 0, 14]]
0.2314	41.65	1000	0.1075	1.0	1.0	1.0	31	1.0	1.0	1.0	30	1.0	1.0	1.0	22	1.0	1.0	1.0	15	1.0	1.0	1.0	1.0	98	1.0	1.0	1.0	98	0.1378	[[0, 1, 2, 3], [0, 31, 0, 0, 0], [1, 0, 30, 0, 0], [2, 0, 0, 22, 0], [3, 0, 0, 0, 15]]
0.1292	45.82	1100	0.0855	1.0	1.0	1.0	31	0.9677	1.0	0.9836	30	1.0	1.0	1.0	22	1.0	0.9333	0.9655	15	0.9898	0.9919	0.9833	0.9873	98	0.9901	0.9898	0.9897	98	0.1038	[[0, 1, 2, 3], [0, 31, 0, 0, 0], [1, 0, 30, 0, 0], [2, 0, 0, 22, 0], [3, 0, 1, 0, 14]]
0.0809	49.98	1200	0.1364	1.0	0.9677	0.9836	31	0.9355	0.9667	0.9508	30	0.9565	1.0	0.9778	22	1.0	0.9333	0.9655	15	0.9694	0.9730	0.9669	0.9694	98	0.9705	0.9694	0.9695	98	0.1309	[[0, 1, 2, 3], [0, 30, 1, 0, 0], [1, 0, 29, 1, 0], [2, 0, 0, 22, 0], [3, 0, 1, 0, 14]]
0.0605	54.16	1300	0.0987	1.0	0.9677	0.9836	31	0.9375	1.0	0.9677	30	1.0	1.0	1.0	22	1.0	0.9333	0.9655	15	0.9796	0.9844	0.9753	0.9792	98	0.9809	0.9796	0.9797	98	0.1073	[[0, 1, 2, 3], [0, 30, 1, 0, 0], [1, 0, 30, 0, 0], [2, 0, 0, 22, 0], [3, 0, 1, 0, 14]]
0.0558	58.33	1400	0.0994	1.0	0.9677	0.9836	31	0.9355	0.9667	0.9508	30	0.9565	1.0	0.9778	22	1.0	0.9333	0.9655	15	0.9694	0.9730	0.9669	0.9694	98	0.9705	0.9694	0.9695	98	0.1048	[[0, 1, 2, 3], [0, 30, 1, 0, 0], [1, 0, 29, 1, 0], [2, 0, 0, 22, 0], [3, 0, 1, 0, 14]]
0.038	62.49	1500	0.0666	1.0	0.9677	0.9836	31	0.9375	1.0	0.9677	30	1.0	1.0	1.0	22	1.0	0.9333	0.9655	15	0.9796	0.9844	0.9753	0.9792	98	0.9809	0.9796	0.9797	98	0.0979	[[0, 1, 2, 3], [0, 30, 1, 0, 0], [1, 0, 30, 0, 0], [2, 0, 0, 22, 0], [3, 0, 1, 0, 14]]
0.0415	66.65	1600	0.0938	1.0	0.9677	0.9836	31	0.9355	0.9667	0.9508	30	0.9565	1.0	0.9778	22	1.0	0.9333	0.9655	15	0.9694	0.9730	0.9669	0.9694	98	0.9705	0.9694	0.9695	98	0.1004	[[0, 1, 2, 3], [0, 30, 1, 0, 0], [1, 0, 29, 1, 0], [2, 0, 0, 22, 0], [3, 0, 1, 0, 14]]

Framework versions

Transformers 4.25.1
Pytorch 1.13.0+cu116
Datasets 2.8.0
Tokenizers 0.13.2

model_phoneme_onSet1

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

NSDT 3DConvert

UnrealSynth

DreamTexture.js