ratish/DBERT_CleanDesc_v2

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

Model description

More information needed

More information needed

More information needed

The following hyperparameters were used during training:

optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 6180, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
training_precision: float32

Train Loss	Validation Loss	Train Accuracy	Epoch
2.2247	2.0414	0.375	0
1.6722	1.6034	0.575	1
1.2412	1.3270	0.6	2
0.9495	1.0999	0.6	3
0.7464	0.9892	0.65	4
0.6087	0.8445	0.75	5
0.4628	0.8918	0.7	6
0.3747	0.7971	0.775	7
0.3069	0.7776	0.75	8
0.2492	0.6877	0.825	9
0.2148	0.7085	0.8	10
0.1793	0.6896	0.85	11
0.1598	0.7230	0.85	12
0.1308	0.7365	0.85	13
0.1211	0.6985	0.85	14
0.1023	0.6592	0.85	15
0.0892	0.6621	0.85	16
0.0885	0.6387	0.85	17
0.0738	0.6606	0.85	18