Hyperparameters:
- learning rate: 2e-5
- weight decay: 0.01
- per_device_train_batch_size: 8
- per_device_eval_batch_size: 8
- gradient_accumulation_steps:1
- eval steps: 24000
- max_length: 512
- num_epochs: 2
- hidden_dropout_prob: 0.3
- attention_probs_dropout_prob: 0.25
Dataset version:
- taskydata/10xp3nirstbbflanse_5xc4
Checkpoint:
- 48000 steps
Results on Validation set:
| Step | Training Loss | Validation Loss | Accuracy | Precision | Recall | F1 |
|---|---|---|---|---|---|---|
| 24000 | 0.052000 | 0.071572 | 0.988261 | 0.999752 | 0.987852 | 0.993767 |
| 48000 | 0.015100 | 0.026952 | 0.995925 | 0.999564 | 0.996132 | 0.997846 |
Wandb logs:
- https://wandb.ai/manandey/huggingface/runs/2vh7iwi6?workspace=user-manandey
