<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->
out
This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on the None dataset. It achieves the following results on the evaluation set:
- Loss: 2.5070
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 10
- num_epochs: 10
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.1061 | 0.07 | 20 | 1.0384 |
| 1.1046 | 0.14 | 40 | 1.0482 |
| 0.9288 | 0.21 | 60 | 1.0336 |
| 0.9333 | 0.28 | 80 | 1.0169 |
| 0.8578 | 0.36 | 100 | 1.0229 |
| 0.8732 | 0.43 | 120 | 1.0262 |
| 0.8619 | 0.5 | 140 | 1.0202 |
| 0.9425 | 0.57 | 160 | 1.0042 |
| 1.0037 | 0.64 | 180 | 1.0076 |
| 0.7348 | 0.71 | 200 | 1.0183 |
| 0.9262 | 0.78 | 220 | 1.0073 |
| 0.8419 | 0.85 | 240 | 1.0219 |
| 0.8592 | 0.92 | 260 | 1.0108 |
| 1.1504 | 1.0 | 280 | 1.0239 |
| 0.3909 | 1.07 | 300 | 1.1543 |
| 0.2493 | 1.14 | 320 | 1.2198 |
| 0.2784 | 1.21 | 340 | 1.1838 |
| 0.3508 | 1.28 | 360 | 1.1628 |
| 0.3395 | 1.35 | 380 | 1.1917 |
| 0.4068 | 1.42 | 400 | 1.1992 |
| 0.2573 | 1.49 | 420 | 1.2275 |
| 0.3582 | 1.56 | 440 | 1.1592 |
| 0.3871 | 1.64 | 460 | 1.1989 |
| 0.4219 | 1.71 | 480 | 1.2173 |
| 0.2557 | 1.78 | 500 | 1.1983 |
| 0.2818 | 1.85 | 520 | 1.1805 |
| 0.3541 | 1.92 | 540 | 1.2021 |
| 0.3196 | 1.99 | 560 | 1.2108 |
| 0.0431 | 2.06 | 580 | 1.5894 |
| 0.0692 | 2.13 | 600 | 1.5097 |
| 0.1278 | 2.2 | 620 | 1.4193 |
| 0.0402 | 2.28 | 640 | 1.5403 |
| 0.0824 | 2.35 | 660 | 1.4192 |
| 0.0726 | 2.42 | 680 | 1.4931 |
| 0.0285 | 2.49 | 700 | 1.5074 |
| 0.0563 | 2.56 | 720 | 1.4607 |
| 0.0481 | 2.63 | 740 | 1.5027 |
| 0.0567 | 2.7 | 760 | 1.4122 |
| 0.0863 | 2.77 | 780 | 1.4192 |
| 0.1338 | 2.84 | 800 | 1.4757 |
| 0.0293 | 2.92 | 820 | 1.5383 |
| 0.1264 | 2.99 | 840 | 1.5248 |
| 0.0097 | 3.06 | 860 | 1.6430 |
| 0.0451 | 3.13 | 880 | 1.8428 |
| 0.0517 | 3.2 | 900 | 1.6281 |
| 0.0223 | 3.27 | 920 | 1.6763 |
| 0.0244 | 3.34 | 940 | 1.7325 |
| 0.0111 | 3.41 | 960 | 1.6546 |
| 0.0356 | 3.48 | 980 | 1.6840 |
| 0.0234 | 3.56 | 1000 | 1.6053 |
| 0.0226 | 3.63 | 1020 | 1.6186 |
| 0.033 | 3.7 | 1040 | 1.7194 |
| 0.048 | 3.77 | 1060 | 1.7855 |
| 0.062 | 3.84 | 1080 | 1.6502 |
| 0.0129 | 3.91 | 1100 | 1.6457 |
| 0.0153 | 3.98 | 1120 | 1.6681 |
| 0.0503 | 4.05 | 1140 | 1.9035 |
| 0.0065 | 4.12 | 1160 | 2.1109 |
| 0.0165 | 4.2 | 1180 | 1.9769 |
| 0.0137 | 4.27 | 1200 | 1.8834 |
| 0.0099 | 4.34 | 1220 | 1.8276 |
| 0.0089 | 4.41 | 1240 | 1.8853 |
| 0.0189 | 4.48 | 1260 | 1.9655 |
| 0.0013 | 4.55 | 1280 | 2.0094 |
| 0.0095 | 4.62 | 1300 | 2.0077 |
| 0.0311 | 4.69 | 1320 | 2.0023 |
| 0.0275 | 4.76 | 1340 | 1.9541 |
| 0.0098 | 4.84 | 1360 | 2.0180 |
| 0.007 | 4.91 | 1380 | 1.9157 |
| 0.005 | 4.98 | 1400 | 2.0126 |
| 0.0002 | 5.05 | 1420 | 2.1353 |
| 0.0026 | 5.12 | 1440 | 2.1879 |
| 0.0019 | 5.19 | 1460 | 2.2352 |
| 0.029 | 5.26 | 1480 | 2.2655 |
| 0.0042 | 5.33 | 1500 | 2.2235 |
| 0.0006 | 5.4 | 1520 | 2.2615 |
| 0.0002 | 5.48 | 1540 | 2.3467 |
| 0.0004 | 5.55 | 1560 | 2.3185 |
| 0.001 | 5.62 | 1580 | 2.2561 |
| 0.0014 | 5.69 | 1600 | 2.2808 |
| 0.0003 | 5.76 | 1620 | 2.3172 |
| 0.0018 | 5.83 | 1640 | 2.3590 |
| 0.0449 | 5.9 | 1660 | 2.3664 |
| 0.001 | 5.97 | 1680 | 2.3406 |
| 0.0003 | 6.04 | 1700 | 2.3714 |
| 0.0005 | 6.12 | 1720 | 2.3949 |
| 0.0001 | 6.19 | 1740 | 2.4272 |
| 0.0001 | 6.26 | 1760 | 2.4392 |
| 0.0002 | 6.33 | 1780 | 2.4290 |
| 0.0002 | 6.4 | 1800 | 2.4404 |
| 0.0001 | 6.47 | 1820 | 2.4518 |
| 0.0001 | 6.54 | 1840 | 2.4554 |
| 0.0001 | 6.61 | 1860 | 2.4554 |
| 0.0002 | 6.68 | 1880 | 2.4584 |
| 0.0001 | 6.76 | 1900 | 2.4634 |
| 0.0002 | 6.83 | 1920 | 2.4726 |
| 0.0002 | 6.9 | 1940 | 2.4746 |
| 0.0001 | 6.97 | 1960 | 2.4763 |
| 0.0 | 7.04 | 1980 | 2.4815 |
| 0.0001 | 7.11 | 2000 | 2.4806 |
| 0.0001 | 7.18 | 2020 | 2.4837 |
| 0.0001 | 7.25 | 2040 | 2.4874 |
| 0.0001 | 7.32 | 2060 | 2.4866 |
| 0.0001 | 7.4 | 2080 | 2.4894 |
| 0.0001 | 7.47 | 2100 | 2.4931 |
| 0.0001 | 7.54 | 2120 | 2.4955 |
| 0.0001 | 7.61 | 2140 | 2.4950 |
| 0.0001 | 7.68 | 2160 | 2.4980 |
| 0.0001 | 7.75 | 2180 | 2.4982 |
| 0.0001 | 7.82 | 2200 | 2.4985 |
| 0.0 | 7.89 | 2220 | 2.5018 |
| 0.0001 | 7.96 | 2240 | 2.5037 |
| 0.0002 | 8.04 | 2260 | 2.5048 |
| 0.0001 | 8.11 | 2280 | 2.5038 |
| 0.0001 | 8.18 | 2300 | 2.5037 |
| 0.0001 | 8.25 | 2320 | 2.5033 |
| 0.0001 | 8.32 | 2340 | 2.5048 |
| 0.0 | 8.39 | 2360 | 2.5055 |
| 0.0001 | 8.46 | 2380 | 2.5068 |
| 0.0001 | 8.53 | 2400 | 2.5045 |
| 0.0001 | 8.6 | 2420 | 2.5058 |
| 0.0001 | 8.68 | 2440 | 2.5041 |
| 0.0001 | 8.75 | 2460 | 2.5042 |
| 0.0001 | 8.82 | 2480 | 2.5052 |
| 0.0001 | 8.89 | 2500 | 2.5051 |
| 0.0001 | 8.96 | 2520 | 2.5068 |
| 0.0001 | 9.03 | 2540 | 2.5057 |
| 0.0001 | 9.1 | 2560 | 2.5057 |
| 0.0001 | 9.17 | 2580 | 2.5079 |
| 0.0001 | 9.24 | 2600 | 2.5061 |
| 0.0001 | 9.32 | 2620 | 2.5069 |
| 0.0001 | 9.39 | 2640 | 2.5062 |
| 0.0002 | 9.46 | 2660 | 2.5072 |
| 0.0001 | 9.53 | 2680 | 2.5067 |
| 0.0001 | 9.6 | 2700 | 2.5061 |
| 0.0001 | 9.67 | 2720 | 2.5069 |
| 0.0001 | 9.74 | 2740 | 2.5052 |
| 0.0001 | 9.81 | 2760 | 2.5053 |
| 0.0001 | 9.88 | 2780 | 2.5064 |
| 0.0001 | 9.96 | 2800 | 2.5070 |
Framework versions
- Transformers 4.35.0.dev0
- Pytorch 2.0.1+cu118
- Datasets 2.14.5
- Tokenizers 0.14.1
