<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
legal_bert_sm_gen1_cv_4
This model is a fine-tuned version of wiorz/legal_bert_sm_gen1 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 1.9479
- Accuracy: 0.832
- Precision: 0.6709
- Recall: 0.2718
- F1: 0.3869
- D-index: 1.5692
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 8000
- num_epochs: 20
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1 | D-index |
|---|---|---|---|---|---|---|---|---|
| No log | 1.0 | 250 | 2.0287 | 0.827 | 0.5965 | 0.3487 | 0.4401 | 1.5883 |
| 1.656 | 2.0 | 500 | 0.7873 | 0.822 | 0.5556 | 0.4359 | 0.4885 | 1.6103 |
| 1.656 | 3.0 | 750 | 0.4216 | 0.834 | 0.6706 | 0.2923 | 0.4071 | 1.5788 |
| 0.3927 | 4.0 | 1000 | 0.4208 | 0.83 | 0.6033 | 0.3744 | 0.4620 | 1.6007 |
| 0.3927 | 5.0 | 1250 | 0.5522 | 0.832 | 0.5985 | 0.4205 | 0.4940 | 1.6185 |
| 0.2162 | 6.0 | 1500 | 0.6989 | 0.83 | 0.6016 | 0.3795 | 0.4654 | 1.6024 |
| 0.2162 | 7.0 | 1750 | 1.1381 | 0.826 | 0.5778 | 0.4 | 0.4727 | 1.6039 |
| 0.0856 | 8.0 | 2000 | 1.5135 | 0.831 | 0.6477 | 0.2923 | 0.4028 | 1.5748 |
| 0.0856 | 9.0 | 2250 | 1.6362 | 0.837 | 0.6667 | 0.3282 | 0.4399 | 1.5948 |
| 0.0378 | 10.0 | 2500 | 1.6518 | 0.839 | 0.6545 | 0.3692 | 0.4721 | 1.6110 |
| 0.0378 | 11.0 | 2750 | 1.7085 | 0.833 | 0.6186 | 0.3744 | 0.4665 | 1.6047 |
| 0.0226 | 12.0 | 3000 | 1.7801 | 0.832 | 0.6063 | 0.3949 | 0.4783 | 1.6101 |
| 0.0226 | 13.0 | 3250 | 1.7308 | 0.83 | 0.5887 | 0.4256 | 0.4940 | 1.6176 |
| 0.0218 | 14.0 | 3500 | 1.9745 | 0.829 | 0.6176 | 0.3231 | 0.4242 | 1.5824 |
| 0.0218 | 15.0 | 3750 | 1.8087 | 0.832 | 0.608 | 0.3897 | 0.475 | 1.6085 |
| 0.0316 | 16.0 | 4000 | 1.7999 | 0.826 | 0.5814 | 0.3846 | 0.4630 | 1.5988 |
| 0.0316 | 17.0 | 4250 | 1.9195 | 0.828 | 0.6095 | 0.3282 | 0.4267 | 1.5828 |
| 0.0266 | 18.0 | 4500 | 1.9005 | 0.825 | 0.5926 | 0.3282 | 0.4224 | 1.5788 |
| 0.0266 | 19.0 | 4750 | 1.8568 | 0.826 | 0.5814 | 0.3846 | 0.4630 | 1.5988 |
| 0.0243 | 20.0 | 5000 | 1.9479 | 0.832 | 0.6709 | 0.2718 | 0.3869 | 1.5692 |
Framework versions
- Transformers 4.28.0
- Pytorch 2.0.1+cu118
- Datasets 2.12.0
- Tokenizers 0.13.3
