model_phoneme_onSet0.0

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.0340
0 Precision: 1.0
0 Recall: 1.0
0 F1-score: 1.0
0 Support: 27
1 Precision: 1.0
1 Recall: 1.0
1 F1-score: 1.0
1 Support: 31
2 Precision: 1.0
2 Recall: 1.0
2 F1-score: 1.0
2 Support: 24
3 Precision: 1.0
3 Recall: 1.0
3 F1-score: 1.0
3 Support: 16
Accuracy: 1.0
Macro avg Precision: 1.0
Macro avg Recall: 1.0
Macro avg F1-score: 1.0
Macro avg Support: 98
Weighted avg Precision: 1.0
Weighted avg Recall: 1.0
Weighted avg F1-score: 1.0
Weighted avg Support: 98
Wer: 0.0612
Mtrix: [[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 31, 0, 0], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 8
eval_batch_size: 8
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 16
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 200
num_epochs: 70
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	0 Precision	0 Recall	0 F1-score	0 Support	1 Precision	1 Recall	1 F1-score	1 Support	2 Precision	2 Recall	2 F1-score	2 Support	3 Precision	3 Recall	3 F1-score	3 Support	Accuracy	Macro avg Precision	Macro avg Recall	Macro avg F1-score	Macro avg Support	Weighted avg Precision	Weighted avg Recall	Weighted avg F1-score	Weighted avg Support	Wer	Mtrix
4.0755	4.16	100	3.4544	1.0	0.2963	0.4571	27	0.0	0.0	0.0	31	0.0	0.0	0.0	24	0.1778	1.0	0.3019	16	0.2449	0.2944	0.3241	0.1898	98	0.3045	0.2449	0.1752	98	0.9965	[[0, 1, 2, 3], [0, 8, 0, 0, 19], [1, 0, 0, 0, 31], [2, 0, 0, 0, 24], [3, 0, 0, 0, 16]]
3.3477	8.33	200	3.1963	1.0	0.2963	0.4571	27	0.0	0.0	0.0	31	0.0	0.0	0.0	24	0.1778	1.0	0.3019	16	0.2449	0.2944	0.3241	0.1898	98	0.3045	0.2449	0.1752	98	0.9965	[[0, 1, 2, 3], [0, 8, 0, 0, 19], [1, 0, 0, 0, 31], [2, 0, 0, 0, 24], [3, 0, 0, 0, 16]]
3.16	12.49	300	3.1744	1.0	0.2963	0.4571	27	0.0	0.0	0.0	31	0.0	0.0	0.0	24	0.1778	1.0	0.3019	16	0.2449	0.2944	0.3241	0.1898	98	0.3045	0.2449	0.1752	98	0.9965	[[0, 1, 2, 3], [0, 8, 0, 0, 19], [1, 0, 0, 0, 31], [2, 0, 0, 0, 24], [3, 0, 0, 0, 16]]
3.0366	16.65	400	3.0466	1.0	0.2963	0.4571	27	0.0	0.0	0.0	31	0.0	0.0	0.0	24	0.1778	1.0	0.3019	16	0.2449	0.2944	0.3241	0.1898	98	0.3045	0.2449	0.1752	98	0.9965	[[0, 1, 2, 3], [0, 8, 0, 0, 19], [1, 0, 0, 0, 31], [2, 0, 0, 0, 24], [3, 0, 0, 0, 16]]
2.6349	20.82	500	2.4959	0.6429	1.0	0.7826	27	0.5185	0.4516	0.4828	31	0.625	0.4167	0.5	24	0.9231	0.75	0.8276	16	0.6429	0.6774	0.6546	0.6482	98	0.6449	0.6429	0.6259	98	0.9809	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 14, 14, 3, 0], [2, 1, 12, 10, 1], [3, 0, 1, 3, 12]]
2.1268	24.98	600	2.0605	1.0	0.8148	0.8980	27	0.7188	0.7419	0.7302	31	0.6667	0.8333	0.7407	24	1.0	0.875	0.9333	16	0.8061	0.8464	0.8163	0.8255	98	0.8294	0.8061	0.8122	98	0.9729	[[0, 1, 2, 3], [0, 22, 5, 0, 0], [1, 0, 23, 8, 0], [2, 0, 4, 20, 0], [3, 0, 0, 2, 14]]
1.7548	29.16	700	1.5829	1.0	1.0	1.0	27	1.0	0.9355	0.9667	31	0.96	1.0	0.9796	24	0.9412	1.0	0.9697	16	0.9796	0.9753	0.9839	0.9790	98	0.9806	0.9796	0.9795	98	0.9413	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 29, 1, 1], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]
1.3546	33.33	800	1.1662	1.0	1.0	1.0	27	0.9688	1.0	0.9841	31	1.0	0.9583	0.9787	24	1.0	1.0	1.0	16	0.9898	0.9922	0.9896	0.9907	98	0.9901	0.9898	0.9898	98	0.8916	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 31, 0, 0], [2, 0, 1, 23, 0], [3, 0, 0, 0, 16]]
0.8917	37.49	900	0.7394	1.0	0.9630	0.9811	27	0.9688	1.0	0.9841	31	1.0	1.0	1.0	24	1.0	1.0	1.0	16	0.9898	0.9922	0.9907	0.9913	98	0.9901	0.9898	0.9898	98	0.8323	[[0, 1, 2, 3], [0, 26, 1, 0, 0], [1, 0, 31, 0, 0], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]
0.5059	41.65	1000	0.4234	1.0	1.0	1.0	27	1.0	0.9677	0.9836	31	0.96	1.0	0.9796	24	1.0	1.0	1.0	16	0.9898	0.99	0.9919	0.9908	98	0.9902	0.9898	0.9898	98	0.4814	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 30, 1, 0], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]
0.2618	45.82	1100	0.1749	1.0	1.0	1.0	27	1.0	0.9677	0.9836	31	0.96	1.0	0.9796	24	1.0	1.0	1.0	16	0.9898	0.99	0.9919	0.9908	98	0.9902	0.9898	0.9898	98	0.1576	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 30, 1, 0], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]
0.126	49.98	1200	0.1227	1.0	1.0	1.0	27	1.0	0.9355	0.9667	31	0.96	1.0	0.9796	24	0.9412	1.0	0.9697	16	0.9796	0.9753	0.9839	0.9790	98	0.9806	0.9796	0.9795	98	0.0989	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 29, 1, 1], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]
0.1138	54.16	1300	0.0469	1.0	1.0	1.0	27	1.0	1.0	1.0	31	1.0	1.0	1.0	24	1.0	1.0	1.0	16	1.0	1.0	1.0	1.0	98	1.0	1.0	1.0	98	0.0693	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 31, 0, 0], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]
0.0675	58.33	1400	0.0397	1.0	1.0	1.0	27	1.0	1.0	1.0	31	1.0	1.0	1.0	24	1.0	1.0	1.0	16	1.0	1.0	1.0	1.0	98	1.0	1.0	1.0	98	0.0658	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 31, 0, 0], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]
0.0462	62.49	1500	0.0333	1.0	1.0	1.0	27	1.0	1.0	1.0	31	1.0	1.0	1.0	24	1.0	1.0	1.0	16	1.0	1.0	1.0	1.0	98	1.0	1.0	1.0	98	0.0612	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 31, 0, 0], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]
0.0359	66.65	1600	0.0340	1.0	1.0	1.0	27	1.0	1.0	1.0	31	1.0	1.0	1.0	24	1.0	1.0	1.0	16	1.0	1.0	1.0	1.0	98	1.0	1.0	1.0	98	0.0612	[[0, 1, 2, 3], [0, 27, 0, 0, 0], [1, 0, 31, 0, 0], [2, 0, 0, 24, 0], [3, 0, 0, 0, 16]]

Framework versions

Transformers 4.25.1
Pytorch 1.13.0+cu116
Datasets 2.8.0
Tokenizers 0.13.2

model_phoneme_onSet0.0

Model description

Intended uses & limitations

Training and evaluation data

Training procedure

Training hyperparameters

Training results

Framework versions

NSDT 3DConvert

UnrealSynth

DreamTexture.js