<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
neuroscience-to-dev-bio
This model is a fine-tuned version of facebook/bart-large on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.0374
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 128
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 5000
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 19.3369 | 0.95 | 7 | 17.9500 |
| 17.1316 | 1.95 | 14 | 15.1026 |
| 14.0654 | 2.95 | 21 | 12.5013 |
| 12.6374 | 3.95 | 28 | 11.3803 |
| 11.7608 | 4.95 | 35 | 10.2692 |
| 10.5271 | 5.95 | 42 | 8.7652 |
| 9.0429 | 6.95 | 49 | 6.8763 |
| 7.4963 | 7.95 | 56 | 5.9052 |
| 6.6044 | 8.95 | 63 | 5.3443 |
| 6.007 | 9.95 | 70 | 4.8687 |
| 5.4706 | 10.95 | 77 | 4.3708 |
| 4.8812 | 11.95 | 84 | 3.8094 |
| 4.2359 | 12.95 | 91 | 3.1743 |
| 3.4727 | 13.95 | 98 | 2.4480 |
| 2.6582 | 14.95 | 105 | 1.6751 |
| 1.8084 | 15.95 | 112 | 0.9828 |
| 1.0742 | 16.95 | 119 | 0.5074 |
| 0.5521 | 17.95 | 126 | 0.2471 |
| 0.263 | 18.95 | 133 | 0.1276 |
| 0.1281 | 19.95 | 140 | 0.0761 |
| 0.0826 | 20.95 | 147 | 0.0620 |
| 0.0419 | 21.95 | 154 | 0.0434 |
| 0.0685 | 22.95 | 161 | 0.1522 |
| 0.1332 | 23.95 | 168 | 0.0536 |
| 0.0405 | 24.95 | 175 | 0.0405 |
| 0.0214 | 25.95 | 182 | 0.0380 |
| 0.0142 | 26.95 | 189 | 0.0370 |
| 0.0202 | 27.95 | 196 | 0.0375 |
| 0.0105 | 28.95 | 203 | 0.0413 |
| 0.0092 | 29.95 | 210 | 0.0370 |
| 0.0083 | 30.95 | 217 | 0.0384 |
| 0.0079 | 31.95 | 224 | 0.0406 |
| 0.0381 | 32.95 | 231 | 0.0371 |
| 0.011 | 33.95 | 238 | 0.0439 |
| 0.0066 | 34.95 | 245 | 0.0374 |
Framework versions
- Transformers 4.21.2
- Pytorch 1.12.1+cu113
- Datasets 2.4.0
- Tokenizers 0.12.1
