news_classification_model
This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:
- Train Loss: 0.0865
- Train Accuracy: 0.9712
- Validation Loss: 1.5070
- Validation Accuracy: 0.7252
- Epoch: 1
Model description
Output Labels of this Model are:
- 'HEALTH & MEDICINE': 0,
- 'ENTERTAINMENT': 1,
- 'FINANCE': 2,
- 'SCIENCE & TECHNOLOGY': 3,
- 'WORLD NEWS': 4,
- 'EDUCATION': 5,
- 'SOCIETY': 6,
- 'POLITICS': 7,
- 'ENVIRONMENT': 8,
- 'BUSINESS': 9,
- 'MEDIA': 10,
- 'FOOD': 11,
- 'CRIME': 12,
- 'LIFESTYLE': 13,
- 'SPORTS': 14
Intended uses & limitations
This models classifies News headlines into different categories mentioned above.
Training and evaluation data
Model was trained and evaluated using the dataset available here: https://www.kaggle.com/datasets/rmisra/news-category-dataset
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
-
optimizer: {'name': 'Adam',
-
'weight_decay': None,
-
'clipnorm': None,
-
'global_clipnorm': None,
-
'clipvalue': None,
-
'use_ema': False,
-
'ema_momentum': 0.99,
-
'ema_overwrite_frequency': None,
-
'jit_compile': True,
-
'is_legacy_optimizer': False,
-
'learning_rate': 5e-05,
-
'beta_1': 0.9,
-
'beta_2': 0.999,
-
'epsilon': 1e-07,
-
'amsgrad': False}
-
training_precision: float32
Training results
Upon the second round of trainning, the results were as follows:
| Train Loss | Train Accuracy | Validation Loss | Validation Accuracy | Epoch |
|---|---|---|---|---|
| 0.1022 | 0.9666 | 1.4192 | 0.7276 | 0 |
| 0.0865 | 0.9712 | 1.5070 | 0.7252 | 1 |
Framework versions
- Transformers 4.33.1
- TensorFlow 2.13.0
- Tokenizers 0.13.3
