emotion-advance-classifier
This model is a fine-tuned version of distilbert-base-uncased on the emotion dataset. It achieves the following results on the evaluation set:
- Loss: 0.1684
- Accuracy: 0.9325
- F1: 0.9328
This model is trained and evaluated using 'emotion' dataset. A great dataset from an article that explored how emotions are represented in English Twitter messages. Unlike most sentiment analysis datasets that involve just “positive” and “negative” polarities, this dataset con‐ tains six basic emotions: anger, love, fear, joy, sadness, and surprise.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 2
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 |
|---|---|---|---|---|---|
| 0.1757 | 1.0 | 250 | 0.1891 | 0.93 | 0.9309 |
| 0.115 | 2.0 | 500 | 0.1684 | 0.9325 | 0.9328 |
Framework versions
- Transformers 4.31.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.4
- Tokenizers 0.13.3