srimoyee12/my_awesome_model
This model is a fine-tuned version of distilbert-base-uncased on the Auditor Review Dataset. It achieves the following results on the evaluation set:
- Train Loss: 0.1735
- Validation Loss: 0.3834
- Train Accuracy: 0.8524
- Epoch: 3
Model description
This is a simple classifier model based on DistilBERT. It classifies given data into Negative, Neutral or Positive based on the sentiment.
Intended uses & limitations
Can be used for text classification.
This is created for illustration purposes and might not have the highest accuracy.
Training and evaluation data
Default split from the dataset card
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 1210, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
- training_precision: float32
Training results
| Train Loss | Validation Loss | Train Accuracy | Epoch |
|---|---|---|---|
| 0.5919 | 0.4004 | 0.8359 | 0 |
| 0.2881 | 0.3590 | 0.8473 | 1 |
| 0.1735 | 0.3834 | 0.8524 | 2 |
Framework versions
- Transformers 4.27.3
- TensorFlow 2.11.0
- Datasets 2.10.1
- Tokenizers 0.13.2