<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
Pegasus-x-base-govreport-12288-1024-numepoch-5-lr-0.002
This model is a fine-tuned version of google/pegasus-x-base on the govreport-summarization dataset. It achieves the following results on the evaluation set:
- Loss: 1.5935
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.002
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 2.953 | 0.07 | 20 | 2.3208 |
| 2.2294 | 0.15 | 40 | 1.9354 |
| 2.0682 | 0.22 | 60 | 1.8995 |
| 2.0892 | 0.29 | 80 | 1.8481 |
| 2.0304 | 0.37 | 100 | 1.8098 |
| 2.014 | 0.44 | 120 | 1.8216 |
| 1.9686 | 0.51 | 140 | 1.7909 |
| 1.9704 | 0.58 | 160 | 1.8038 |
| 1.9512 | 0.66 | 180 | 1.7712 |
| 1.9458 | 0.73 | 200 | 1.7784 |
| 1.9429 | 0.8 | 220 | 1.7472 |
| 1.9543 | 0.88 | 240 | 1.7330 |
| 1.9124 | 0.95 | 260 | 1.7163 |
| 1.8301 | 1.02 | 280 | 1.7130 |
| 1.7666 | 1.1 | 300 | 1.7080 |
| 1.7531 | 1.17 | 320 | 1.7076 |
| 1.7344 | 1.24 | 340 | 1.7031 |
| 1.7538 | 1.32 | 360 | 1.7103 |
| 1.7472 | 1.39 | 380 | 1.6936 |
| 1.7573 | 1.46 | 400 | 1.6819 |
| 1.7675 | 1.53 | 420 | 1.6885 |
| 1.7501 | 1.61 | 440 | 1.6722 |
| 1.7503 | 1.68 | 460 | 1.6620 |
| 1.7245 | 1.75 | 480 | 1.6629 |
| 1.7448 | 1.83 | 500 | 1.6502 |
| 1.7479 | 1.9 | 520 | 1.6579 |
| 1.6973 | 1.97 | 540 | 1.6514 |
| 1.5818 | 2.05 | 560 | 1.6610 |
| 1.5976 | 2.12 | 580 | 1.6420 |
| 1.5851 | 2.19 | 600 | 1.6471 |
| 1.6306 | 2.27 | 620 | 1.6471 |
| 1.5761 | 2.34 | 640 | 1.6452 |
| 1.5824 | 2.41 | 660 | 1.6378 |
| 1.5651 | 2.48 | 680 | 1.6300 |
| 1.6094 | 2.56 | 700 | 1.6325 |
| 1.5866 | 2.63 | 720 | 1.6294 |
| 1.5573 | 2.7 | 740 | 1.6274 |
| 1.5799 | 2.78 | 760 | 1.6255 |
| 1.5801 | 2.85 | 780 | 1.6230 |
| 1.5655 | 2.92 | 800 | 1.6107 |
| 1.5453 | 3.0 | 820 | 1.6126 |
| 1.5005 | 3.07 | 840 | 1.6199 |
| 1.4822 | 3.14 | 860 | 1.6238 |
| 1.4579 | 3.22 | 880 | 1.6168 |
| 1.4897 | 3.29 | 900 | 1.6134 |
| 1.4453 | 3.36 | 920 | 1.6213 |
| 1.4738 | 3.43 | 940 | 1.6111 |
| 1.4912 | 3.51 | 960 | 1.6118 |
| 1.4461 | 3.58 | 980 | 1.6071 |
| 1.4365 | 3.65 | 1000 | 1.6034 |
| 1.4244 | 3.73 | 1020 | 1.6008 |
| 1.4682 | 3.8 | 1040 | 1.5991 |
| 1.4714 | 3.87 | 1060 | 1.5963 |
| 1.4586 | 3.95 | 1080 | 1.5987 |
| 1.4434 | 4.02 | 1100 | 1.6166 |
| 1.3898 | 4.09 | 1120 | 1.6080 |
| 1.4035 | 4.17 | 1140 | 1.6099 |
| 1.4145 | 4.24 | 1160 | 1.6064 |
| 1.3806 | 4.31 | 1180 | 1.6003 |
| 1.3645 | 4.38 | 1200 | 1.6004 |
| 1.381 | 4.46 | 1220 | 1.6053 |
| 1.3606 | 4.53 | 1240 | 1.6004 |
| 1.3788 | 4.6 | 1260 | 1.5990 |
| 1.3526 | 4.68 | 1280 | 1.6029 |
| 1.3618 | 4.75 | 1300 | 1.5941 |
| 1.3684 | 4.82 | 1320 | 1.5949 |
| 1.3721 | 4.9 | 1340 | 1.5948 |
| 1.3695 | 4.97 | 1360 | 1.5935 |
Framework versions
- Transformers 4.30.2
- Pytorch 2.0.1+cu117
- Datasets 2.13.1
- Tokenizers 0.13.3
