<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
distilbert-base-cased-distilled-squad-finetuned-lr3e-06-epochs100
This model is a fine-tuned version of distilbert-base-cased-distilled-squad on the None dataset. It achieves the following results on the evaluation set:
- Loss: 3.3462
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-06
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 1.0 | 9 | 3.0777 |
| No log | 2.0 | 18 | 3.1319 |
| No log | 3.0 | 27 | 3.1694 |
| No log | 4.0 | 36 | 3.0616 |
| No log | 5.0 | 45 | 3.1448 |
| No log | 6.0 | 54 | 3.1118 |
| No log | 7.0 | 63 | 2.9115 |
| No log | 8.0 | 72 | 2.8013 |
| No log | 9.0 | 81 | 3.2407 |
| No log | 10.0 | 90 | 3.2540 |
| No log | 11.0 | 99 | 3.2567 |
| No log | 12.0 | 108 | 3.2652 |
| No log | 13.0 | 117 | 3.2635 |
| No log | 14.0 | 126 | 3.2646 |
| No log | 15.0 | 135 | 3.2717 |
| No log | 16.0 | 144 | 3.2749 |
| No log | 17.0 | 153 | 3.2778 |
| No log | 18.0 | 162 | 3.2802 |
| No log | 19.0 | 171 | 3.2908 |
| No log | 20.0 | 180 | 3.2998 |
| No log | 21.0 | 189 | 3.2963 |
| No log | 22.0 | 198 | 3.2936 |
| No log | 23.0 | 207 | 3.3001 |
| No log | 24.0 | 216 | 3.3034 |
| No log | 25.0 | 225 | 3.3164 |
| No log | 26.0 | 234 | 3.3119 |
| No log | 27.0 | 243 | 3.2937 |
| No log | 28.0 | 252 | 3.2761 |
| No log | 29.0 | 261 | 3.2692 |
| No log | 30.0 | 270 | 3.2687 |
| No log | 31.0 | 279 | 3.3021 |
| No log | 32.0 | 288 | 3.3296 |
| No log | 33.0 | 297 | 3.3427 |
| No log | 34.0 | 306 | 3.3303 |
| No log | 35.0 | 315 | 3.3177 |
| No log | 36.0 | 324 | 3.2875 |
| No log | 37.0 | 333 | 3.2566 |
| No log | 38.0 | 342 | 3.2476 |
| No log | 39.0 | 351 | 3.2495 |
| No log | 40.0 | 360 | 3.2710 |
| No log | 41.0 | 369 | 3.2782 |
| No log | 42.0 | 378 | 3.2670 |
| No log | 43.0 | 387 | 3.2743 |
| No log | 44.0 | 396 | 3.3123 |
| No log | 45.0 | 405 | 3.3308 |
| No log | 46.0 | 414 | 3.3599 |
| No log | 47.0 | 423 | 3.2345 |
| No log | 48.0 | 432 | 3.1899 |
| No log | 49.0 | 441 | 3.2649 |
| No log | 50.0 | 450 | 3.3745 |
| No log | 51.0 | 459 | 3.3841 |
| No log | 52.0 | 468 | 3.3477 |
| No log | 53.0 | 477 | 3.4445 |
| No log | 54.0 | 486 | 3.5382 |
| No log | 55.0 | 495 | 3.5705 |
| 0.0001 | 56.0 | 504 | 3.5022 |
| 0.0001 | 57.0 | 513 | 3.4424 |
| 0.0001 | 58.0 | 522 | 3.4941 |
| 0.0001 | 59.0 | 531 | 3.4784 |
| 0.0001 | 60.0 | 540 | 3.4759 |
| 0.0001 | 61.0 | 549 | 3.4902 |
| 0.0001 | 62.0 | 558 | 3.4874 |
| 0.0001 | 63.0 | 567 | 3.4836 |
| 0.0001 | 64.0 | 576 | 3.4769 |
| 0.0001 | 65.0 | 585 | 3.4762 |
| 0.0001 | 66.0 | 594 | 3.4609 |
| 0.0001 | 67.0 | 603 | 3.4599 |
| 0.0001 | 68.0 | 612 | 3.4530 |
| 0.0001 | 69.0 | 621 | 3.4525 |
| 0.0001 | 70.0 | 630 | 3.4634 |
| 0.0001 | 71.0 | 639 | 3.5287 |
| 0.0001 | 72.0 | 648 | 3.5306 |
| 0.0001 | 73.0 | 657 | 3.5194 |
| 0.0001 | 74.0 | 666 | 3.5239 |
| 0.0001 | 75.0 | 675 | 3.5213 |
| 0.0001 | 76.0 | 684 | 3.4864 |
| 0.0001 | 77.0 | 693 | 3.4199 |
| 0.0001 | 78.0 | 702 | 3.3633 |
| 0.0001 | 79.0 | 711 | 3.3121 |
| 0.0001 | 80.0 | 720 | 3.2907 |
| 0.0001 | 81.0 | 729 | 3.3259 |
| 0.0001 | 82.0 | 738 | 3.3401 |
| 0.0001 | 83.0 | 747 | 3.3613 |
| 0.0001 | 84.0 | 756 | 3.3617 |
| 0.0001 | 85.0 | 765 | 3.3348 |
| 0.0001 | 86.0 | 774 | 3.2861 |
| 0.0001 | 87.0 | 783 | 3.2698 |
| 0.0001 | 88.0 | 792 | 3.3305 |
| 0.0001 | 89.0 | 801 | 3.3725 |
| 0.0001 | 90.0 | 810 | 3.3847 |
| 0.0001 | 91.0 | 819 | 3.3876 |
| 0.0001 | 92.0 | 828 | 3.3788 |
| 0.0001 | 93.0 | 837 | 3.3743 |
| 0.0001 | 94.0 | 846 | 3.3662 |
| 0.0001 | 95.0 | 855 | 3.3581 |
| 0.0001 | 96.0 | 864 | 3.3527 |
| 0.0001 | 97.0 | 873 | 3.3489 |
| 0.0001 | 98.0 | 882 | 3.3471 |
| 0.0001 | 99.0 | 891 | 3.3463 |
| 0.0001 | 100.0 | 900 | 3.3462 |
Framework versions
- Transformers 4.28.1
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.3
