<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
summarise
This model is a fine-tuned version of vinai/bartpho-word-base on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 1.2042
- Rouge1: 0.3976
- Rouge2: 0.1671
- Rougel: 0.2789
- Rougelsum: 0.2783
- Gen Len: 19.7789
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 20
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| No log | 0.99 | 97 | 1.5996 | 0.2912 | 0.1103 | 0.2037 | 0.203 | 15.7121 |
| No log | 2.0 | 195 | 1.3741 | 0.3885 | 0.1547 | 0.2714 | 0.2714 | 19.7044 |
| No log | 2.99 | 292 | 1.3240 | 0.3687 | 0.1505 | 0.2624 | 0.2624 | 18.9743 |
| No log | 4.0 | 390 | 1.2874 | 0.3929 | 0.1611 | 0.2775 | 0.2771 | 19.91 |
| No log | 4.99 | 487 | 1.3065 | 0.3829 | 0.1552 | 0.2667 | 0.2664 | 19.3059 |
| 2.392 | 6.0 | 585 | 1.2555 | 0.3836 | 0.1549 | 0.2689 | 0.2686 | 19.6452 |
| 2.392 | 6.99 | 682 | 1.2605 | 0.3796 | 0.157 | 0.2665 | 0.2663 | 19.3496 |
| 2.392 | 8.0 | 780 | 1.2517 | 0.3969 | 0.1621 | 0.2787 | 0.2781 | 19.8946 |
| 2.392 | 8.99 | 877 | 1.2349 | 0.3885 | 0.1593 | 0.2713 | 0.2709 | 19.7326 |
| 2.392 | 10.0 | 975 | 1.2197 | 0.3864 | 0.1554 | 0.2701 | 0.2695 | 19.4936 |
| 1.2799 | 10.99 | 1072 | 1.2143 | 0.3805 | 0.1565 | 0.2682 | 0.2675 | 19.5141 |
| 1.2799 | 12.0 | 1170 | 1.2111 | 0.391 | 0.1594 | 0.2729 | 0.2725 | 19.6992 |
| 1.2799 | 12.99 | 1267 | 1.1993 | 0.3849 | 0.1568 | 0.2686 | 0.2681 | 19.6632 |
| 1.2799 | 14.0 | 1365 | 1.2196 | 0.3888 | 0.1587 | 0.2733 | 0.273 | 19.6915 |
| 1.2799 | 14.99 | 1462 | 1.1952 | 0.3947 | 0.1611 | 0.2754 | 0.2747 | 19.8021 |
| 1.2178 | 16.0 | 1560 | 1.1982 | 0.3954 | 0.1652 | 0.2778 | 0.2775 | 19.8689 |
| 1.2178 | 16.99 | 1657 | 1.1988 | 0.3957 | 0.1641 | 0.2765 | 0.276 | 19.7121 |
| 1.2178 | 18.0 | 1755 | 1.2020 | 0.3998 | 0.169 | 0.281 | 0.2806 | 19.7815 |
| 1.2178 | 18.99 | 1852 | 1.1995 | 0.3965 | 0.1662 | 0.2782 | 0.2778 | 19.8226 |
| 1.2178 | 19.9 | 1940 | 1.2042 | 0.3976 | 0.1671 | 0.2789 | 0.2783 | 19.7789 |
Framework versions
- Transformers 4.30.0.dev0
- Pytorch 2.0.0
- Datasets 2.12.0
- Tokenizers 0.13.3
