<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
scotus_new
This model is a fine-tuned version of nlpaueb/legal-bert-small-uncased on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.6724
- Accuracy: 0.9164
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 2.2245 | 1.0 | 708 | 1.6712 | 0.4687 |
| 1.6638 | 2.0 | 1416 | 1.1274 | 0.6715 |
| 0.8816 | 3.0 | 2124 | 0.8631 | 0.7496 |
| 0.577 | 4.0 | 2832 | 0.5606 | 0.8458 |
| 0.2505 | 5.0 | 3540 | 0.4815 | 0.8673 |
| 0.1437 | 6.0 | 4248 | 0.4833 | 0.8723 |
| 0.0896 | 7.0 | 4956 | 0.4940 | 0.8783 |
| 0.0275 | 8.0 | 5664 | 0.4783 | 0.8873 |
| 0.0235 | 9.0 | 6372 | 0.5636 | 0.8888 |
| 0.0071 | 10.0 | 7080 | 0.5898 | 0.8858 |
| 0.0053 | 11.0 | 7788 | 0.7321 | 0.8618 |
| 0.0049 | 12.0 | 8496 | 0.5890 | 0.8918 |
| 0.0048 | 13.0 | 9204 | 0.5987 | 0.8998 |
| 0.0035 | 14.0 | 9912 | 0.5803 | 0.9044 |
| 0.0039 | 15.0 | 10620 | 0.6825 | 0.8828 |
| 0.0058 | 16.0 | 11328 | 0.6242 | 0.9009 |
| 0.0031 | 17.0 | 12036 | 0.6940 | 0.8948 |
| 0.0052 | 18.0 | 12744 | 0.7229 | 0.8913 |
| 0.0053 | 19.0 | 13452 | 0.7286 | 0.8948 |
| 0.0029 | 20.0 | 14160 | 0.6685 | 0.9014 |
| 0.0033 | 21.0 | 14868 | 0.8086 | 0.8883 |
| 0.0039 | 22.0 | 15576 | 0.7520 | 0.8973 |
| 0.0031 | 23.0 | 16284 | 0.7527 | 0.8928 |
| 0.004 | 24.0 | 16992 | 0.6379 | 0.9064 |
| 0.0033 | 25.0 | 17700 | 0.7221 | 0.8998 |
| 0.0029 | 26.0 | 18408 | 0.7113 | 0.9069 |
| 0.0021 | 27.0 | 19116 | 0.8536 | 0.8928 |
| 0.0016 | 28.0 | 19824 | 0.7025 | 0.9094 |
| 0.0019 | 29.0 | 20532 | 0.7789 | 0.8978 |
| 0.0015 | 30.0 | 21240 | 0.7080 | 0.9064 |
| 0.0022 | 31.0 | 21948 | 0.7473 | 0.8983 |
| 0.0007 | 32.0 | 22656 | 0.7111 | 0.9029 |
| 0.0009 | 33.0 | 23364 | 0.9103 | 0.8828 |
| 0.0013 | 34.0 | 24072 | 0.7577 | 0.8963 |
| 0.0013 | 35.0 | 24780 | 0.7397 | 0.9044 |
| 0.0006 | 36.0 | 25488 | 0.8032 | 0.8953 |
| 0.0004 | 37.0 | 26196 | 0.6765 | 0.9099 |
| 0.0014 | 38.0 | 26904 | 0.7900 | 0.8953 |
| 0.0004 | 39.0 | 27612 | 0.6943 | 0.9124 |
| 0.0 | 40.0 | 28320 | 0.7733 | 0.9054 |
| 0.0005 | 41.0 | 29028 | 0.7737 | 0.9044 |
| 0.0003 | 42.0 | 29736 | 0.7045 | 0.9134 |
| 0.0 | 43.0 | 30444 | 0.7331 | 0.9084 |
| 0.0 | 44.0 | 31152 | 0.6729 | 0.9144 |
| 0.0001 | 45.0 | 31860 | 0.7082 | 0.9139 |
| 0.0007 | 46.0 | 32568 | 0.7236 | 0.9134 |
| 0.0 | 47.0 | 33276 | 0.6963 | 0.9159 |
| 0.0 | 48.0 | 33984 | 0.6575 | 0.9179 |
| 0.0 | 49.0 | 34692 | 0.6793 | 0.9159 |
| 0.0001 | 50.0 | 35400 | 0.6724 | 0.9164 |
Framework versions
- Transformers 4.28.0
- Pytorch 2.0.1+cu118
- Datasets 2.14.4
- Tokenizers 0.13.3
