<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
juliana-out
This model is a fine-tuned version of Open-Orca/Mistral-7B-OpenOrca on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.9221
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 6e-06
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-05
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- num_epochs: 100
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.4954 | 1.0 | 1 | 2.8026 |
| 1.5239 | 2.0 | 2 | 2.8021 |
| 1.5146 | 2.0 | 3 | 2.8009 |
| 1.4963 | 3.0 | 4 | 2.7995 |
| 1.4918 | 3.0 | 5 | 2.7968 |
| 1.5166 | 4.0 | 6 | 2.7921 |
| 1.4408 | 4.0 | 7 | 2.7847 |
| 1.5545 | 5.0 | 8 | 2.7768 |
| 1.4642 | 5.0 | 9 | 2.7660 |
| 1.4683 | 6.0 | 10 | 2.7515 |
| 1.4576 | 11.0 | 11 | 2.7306 |
| 1.3832 | 12.0 | 12 | 2.7085 |
| 1.382 | 12.0 | 13 | 2.6802 |
| 1.3257 | 13.0 | 14 | 2.6505 |
| 1.297 | 13.0 | 15 | 2.6189 |
| 1.2334 | 14.0 | 16 | 2.5854 |
| 1.175 | 14.0 | 17 | 2.5478 |
| 1.1355 | 15.0 | 18 | 2.5125 |
| 1.0529 | 15.0 | 19 | 2.4767 |
| 1.0127 | 16.0 | 20 | 2.4438 |
| 0.9518 | 16.0 | 21 | 2.4151 |
| 0.9149 | 17.0 | 22 | 2.3918 |
| 0.8801 | 17.0 | 23 | 2.3743 |
| 0.834 | 18.0 | 24 | 2.3623 |
| 0.7861 | 18.0 | 25 | 2.3552 |
| 0.7733 | 19.0 | 26 | 2.3527 |
| 0.7316 | 19.0 | 27 | 2.3532 |
| 0.6849 | 20.0 | 28 | 2.3578 |
| 0.6713 | 20.0 | 29 | 2.3629 |
| 0.6731 | 21.0 | 30 | 2.3656 |
| 0.6617 | 21.0 | 31 | 2.3640 |
| 0.6296 | 22.0 | 32 | 2.3614 |
| 0.6289 | 22.0 | 33 | 2.3610 |
| 0.61 | 23.0 | 34 | 2.3613 |
| 0.6011 | 23.0 | 35 | 2.3614 |
| 0.5876 | 24.0 | 36 | 2.3567 |
| 0.5734 | 24.0 | 37 | 2.3473 |
| 0.566 | 25.0 | 38 | 2.3345 |
| 0.5519 | 25.0 | 39 | 2.3211 |
| 0.5348 | 26.0 | 40 | 2.3110 |
| 0.5276 | 41.0 | 41 | 2.2998 |
| 0.506 | 42.0 | 42 | 2.2893 |
| 0.4882 | 42.0 | 43 | 2.2811 |
| 0.4961 | 43.0 | 44 | 2.2687 |
| 0.4758 | 43.0 | 45 | 2.2583 |
| 0.4491 | 44.0 | 46 | 2.2487 |
| 0.4451 | 44.0 | 47 | 2.2444 |
| 0.4272 | 45.0 | 48 | 2.2445 |
| 0.4128 | 45.0 | 49 | 2.2465 |
| 0.4046 | 46.0 | 50 | 2.2503 |
| 0.3857 | 46.0 | 51 | 2.2509 |
| 0.3738 | 47.0 | 52 | 2.2532 |
| 0.3504 | 47.0 | 53 | 2.2594 |
| 0.3488 | 48.0 | 54 | 2.2659 |
| 0.3196 | 48.0 | 55 | 2.2729 |
| 0.3168 | 49.0 | 56 | 2.2759 |
| 0.2965 | 49.0 | 57 | 2.2781 |
| 0.2734 | 50.0 | 58 | 2.2861 |
| 0.2583 | 50.0 | 59 | 2.2967 |
| 0.2481 | 51.0 | 60 | 2.3044 |
| 0.2245 | 51.0 | 61 | 2.3136 |
| 0.2185 | 52.0 | 62 | 2.3289 |
| 0.1984 | 52.0 | 63 | 2.3489 |
| 0.1793 | 53.0 | 64 | 2.3623 |
| 0.1647 | 53.0 | 65 | 2.3842 |
| 0.1491 | 54.0 | 66 | 2.4092 |
| 0.1265 | 54.0 | 67 | 2.4389 |
| 0.1267 | 55.0 | 68 | 2.4587 |
| 0.1047 | 55.0 | 69 | 2.4943 |
| 0.0928 | 56.0 | 70 | 2.5323 |
| 0.0798 | 56.0 | 71 | 2.5660 |
| 0.0708 | 57.0 | 72 | 2.5854 |
| 0.0601 | 57.0 | 73 | 2.6185 |
| 0.0499 | 58.0 | 74 | 2.6635 |
| 0.0413 | 58.0 | 75 | 2.7063 |
| 0.0371 | 59.0 | 76 | 2.7359 |
| 0.0279 | 59.0 | 77 | 2.7723 |
| 0.0258 | 60.0 | 78 | 2.7960 |
| 0.0213 | 60.0 | 79 | 2.8199 |
| 0.0162 | 61.0 | 80 | 2.8411 |
| 0.0151 | 61.0 | 81 | 2.8649 |
| 0.0117 | 62.0 | 82 | 2.8757 |
| 0.0111 | 62.0 | 83 | 2.8834 |
| 0.0095 | 63.0 | 84 | 2.8873 |
| 0.0095 | 63.0 | 85 | 2.8977 |
| 0.01 | 64.0 | 86 | 2.9051 |
| 0.0083 | 64.0 | 87 | 2.9116 |
| 0.0077 | 65.0 | 88 | 2.9208 |
| 0.006 | 65.0 | 89 | 2.9261 |
| 0.0115 | 66.0 | 90 | 2.9277 |
| 0.0073 | 66.0 | 91 | 2.9276 |
| 0.0086 | 67.0 | 92 | 2.9258 |
| 0.0071 | 67.0 | 93 | 2.9248 |
| 0.0072 | 68.0 | 94 | 2.9236 |
| 0.0064 | 68.0 | 95 | 2.9255 |
| 0.0068 | 69.0 | 96 | 2.9279 |
| 0.0052 | 69.0 | 97 | 2.9291 |
| 0.0084 | 70.0 | 98 | 2.9238 |
| 0.0054 | 70.0 | 99 | 2.9237 |
| 0.0068 | 71.0 | 100 | 2.9221 |
Framework versions
- Transformers 4.35.0.dev0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1
