Trickshotblaster/masked-lm-tpu

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

Model description

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 0.0001, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 0.0001, 'decay_steps': 22325, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'passive_serialization': True}, 'warmup_steps': 1175, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.001}
training_precision: float32

Train Loss	Train Accuracy	Validation Loss	Validation Accuracy	Epoch
5.1057	0.0447	5.1128	0.0445	0
5.1366	0.0445	5.1449	0.0446	1
5.1152	0.0445	5.1034	0.0441	2
5.1297	0.0442	5.0708	0.0449	3
5.1002	0.0448	5.0494	0.0449	4
5.1155	0.0443	5.1304	0.0440	5
5.0949	0.0445	5.1050	0.0441	6
5.0747	0.0451	5.0769	0.0446	7
5.0840	0.0447	5.0664	0.0442	8
5.0828	0.0446	5.1446	0.0438	9
5.0994	0.0444	5.0545	0.0454	10
5.1009	0.0448	5.0466	0.0452	11
5.1028	0.0442	5.0538	0.0458	12
5.0771	0.0447	5.0645	0.0445	13
5.0809	0.0443	5.0884	0.0438	14
5.0831	0.0445	5.0946	0.0450	15
5.0753	0.0444	5.1057	0.0441	16
5.0786	0.0447	5.1212	0.0439	17
5.0674	0.0446	5.0428	0.0453	18
5.0737	0.0447	5.0155	0.0458	19
5.0510	0.0448	5.1046	0.0443	20
5.0815	0.0446	5.0959	0.0441	21
5.0518	0.0450	5.0441	0.0453	22
5.0514	0.0449	5.0835	0.0442	23
5.0400	0.0449	5.0802	0.0444	24
5.0708	0.0446	5.0587	0.0440	25
5.0745	0.0441	5.0809	0.0440	26
5.0467	0.0446	5.0542	0.0444	27
5.0190	0.0449	5.0191	0.0453	28
5.0588	0.0445	5.0495	0.0449	29