distilbert-base-uncased-MC-News
This model is a fine-tuned version of distilbert-base-uncased. It achieves the following results on the evaluation set:
- Loss: 0.0965
- Accuracy: 0.9643
- F1 Score: 0.9640
Model description
This is a multiclass classification model of news articles.
For more information on how it was created, check out the following link: https://github.com/DunnBC22/NLP_Projects/blob/main/Multiclass%20Classification/Multiclass_Classification%20of%20News%20Articles-CNN%20News.ipynb
Intended uses & limitations
This model is intended to demonstrate my ability to solve a complex problem using technology.
Training and evaluation data
Dataset Source: https://www.kaggle.com/datasets/hadasu92/cnn-articles-after-basic-cleaning
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 4
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 Score |
|---|---|---|---|---|---|
| 0.2026 | 1.0 | 525 | 0.0887 | 0.9571 | 0.9569 |
| 0.0641 | 2.0 | 1050 | 0.0823 | 0.9612 | 0.9608 |
| 0.0383 | 3.0 | 1575 | 0.0890 | 0.9621 | 0.9625 |
| 0.0242 | 4.0 | 2100 | 0.0965 | 0.9643 | 0.9640 |
Framework versions
- Transformers 4.21.3
- Pytorch 1.12.1
- Datasets 2.4.0
- Tokenizers 0.12.1
