<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
codet5p-770m-py-codebleu-64-True-1e-06-0.1
This model is a fine-tuned version of Salesforce/codet5p-770m-py on the mbpp dataset. It achieves the following results on the evaluation set:
- Loss: 0.8108
- Codebleu: 0.0867
- Ngram Match Score: 0.0136
- Weighted Ngram Match Score: 0.0422
- Syntax Match Score: 0.1204
- Dataflow Match Score: 0.0824
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-06
- train_batch_size: 6
- eval_batch_size: 6
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 384
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 50
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | Codebleu | Ngram Match Score | Weighted Ngram Match Score | Syntax Match Score | Dataflow Match Score |
---|---|---|---|---|---|---|---|---|
0.9614 | 1.0 | 1 | 0.9113 | 0.0039 | 0.0000 | 0.0000 | 0.0048 | 0.0049 |
0.4928 | 2.0 | 3 | 0.9113 | 0.0039 | 0.0000 | 0.0000 | 0.0048 | 0.0049 |
0.4867 | 3.0 | 5 | 0.9112 | 0.0055 | 0.0000 | 0.0000 | 0.0067 | 0.0070 |
0.4813 | 4.0 | 7 | 0.9111 | 0.0063 | 0.0000 | 0.0002 | 0.0072 | 0.0084 |
0.4794 | 5.0 | 9 | 0.9108 | 0.0065 | 0.0000 | 0.0002 | 0.0072 | 0.0091 |
0.4857 | 6.0 | 11 | 0.9106 | 0.0124 | 0.0000 | 0.0012 | 0.0173 | 0.0133 |
0.4835 | 7.0 | 13 | 0.9095 | 0.0132 | 0.0000 | 0.0022 | 0.0178 | 0.0147 |
0.4902 | 8.0 | 15 | 0.9090 | 0.0199 | 0.0000 | 0.0054 | 0.0246 | 0.0237 |
0.4859 | 9.0 | 17 | 0.9053 | 0.0206 | 0.0000 | 0.0057 | 0.0255 | 0.0244 |
0.4787 | 10.0 | 19 | 0.9041 | 0.0326 | 0.0002 | 0.0152 | 0.0414 | 0.0363 |
0.485 | 11.0 | 21 | 0.9031 | 0.0435 | 0.0008 | 0.0199 | 0.0554 | 0.0482 |
0.4756 | 12.0 | 23 | 0.8915 | 0.0485 | 0.0020 | 0.0221 | 0.0592 | 0.0559 |
0.4774 | 13.0 | 25 | 0.8893 | 0.0629 | 0.0054 | 0.0314 | 0.0804 | 0.0677 |
0.4724 | 14.0 | 27 | 0.8859 | 0.0664 | 0.0055 | 0.0287 | 0.0877 | 0.0698 |
0.4755 | 15.0 | 29 | 0.8832 | 0.0782 | 0.0093 | 0.0346 | 0.1084 | 0.0761 |
0.458 | 16.0 | 31 | 0.8618 | 0.0797 | 0.0102 | 0.0357 | 0.1103 | 0.0775 |
0.4549 | 17.0 | 33 | 0.8586 | 0.0797 | 0.0105 | 0.0363 | 0.1122 | 0.0754 |
0.448 | 18.0 | 35 | 0.8560 | 0.0804 | 0.0109 | 0.0363 | 0.1132 | 0.0761 |
0.45 | 19.0 | 37 | 0.8530 | 0.0805 | 0.0110 | 0.0363 | 0.1132 | 0.0761 |
0.4403 | 20.0 | 39 | 0.8499 | 0.0794 | 0.0111 | 0.0363 | 0.1113 | 0.0754 |
0.4373 | 21.0 | 41 | 0.8345 | 0.0797 | 0.0118 | 0.0385 | 0.1113 | 0.0754 |
0.4208 | 22.0 | 43 | 0.8381 | 0.0816 | 0.0120 | 0.0386 | 0.1132 | 0.0782 |
0.4159 | 23.0 | 45 | 0.8337 | 0.0824 | 0.0122 | 0.0386 | 0.1137 | 0.0796 |
0.4157 | 24.0 | 47 | 0.8264 | 0.0857 | 0.0136 | 0.0422 | 0.1199 | 0.0803 |
0.4101 | 25.0 | 49 | 0.8161 | 0.0850 | 0.0134 | 0.0420 | 0.1190 | 0.0796 |
0.3181 | 25.4 | 50 | 0.8108 | 0.0867 | 0.0136 | 0.0422 | 0.1204 | 0.0824 |
Framework versions
- Transformers 4.30.0.dev0
- Pytorch 2.0.1
- Datasets 2.13.1
- Tokenizers 0.13.3