Update README.md
Browse files
README.md
CHANGED
|
@@ -1,36 +1,9 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
base_model: mistralai/Mistral-7B-v0.1
|
| 4 |
-
tags:
|
| 5 |
-
- generated_from_trainer
|
| 6 |
-
model-index:
|
| 7 |
-
- name: out
|
| 8 |
-
results: []
|
| 9 |
---
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 13 |
-
|
| 14 |
-
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
| 15 |
-
# out
|
| 16 |
-
|
| 17 |
-
This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
|
| 18 |
-
It achieves the following results on the evaluation set:
|
| 19 |
-
- Loss: 6.8300
|
| 20 |
-
|
| 21 |
-
## Model description
|
| 22 |
-
|
| 23 |
-
More information needed
|
| 24 |
-
|
| 25 |
-
## Intended uses & limitations
|
| 26 |
-
|
| 27 |
-
More information needed
|
| 28 |
-
|
| 29 |
-
## Training and evaluation data
|
| 30 |
-
|
| 31 |
-
More information needed
|
| 32 |
-
|
| 33 |
-
## Training procedure
|
| 34 |
|
| 35 |
### Training hyperparameters
|
| 36 |
|
|
@@ -49,30 +22,6 @@ The following hyperparameters were used during training:
|
|
| 49 |
- lr_scheduler_warmup_steps: 10
|
| 50 |
- num_epochs: 3
|
| 51 |
|
| 52 |
-
### Training results
|
| 53 |
-
|
| 54 |
-
| Training Loss | Epoch | Step | Validation Loss |
|
| 55 |
-
|:-------------:|:-----:|:----:|:---------------:|
|
| 56 |
-
| 1.2465 | 0.16 | 20 | 1.1498 |
|
| 57 |
-
| 8.597 | 0.32 | 40 | 8.5500 |
|
| 58 |
-
| 8.0391 | 0.48 | 60 | 7.6701 |
|
| 59 |
-
| 7.2703 | 0.65 | 80 | 7.4684 |
|
| 60 |
-
| 8.1368 | 0.81 | 100 | 7.7743 |
|
| 61 |
-
| 7.3305 | 0.97 | 120 | 7.3737 |
|
| 62 |
-
| 7.0599 | 1.13 | 140 | 7.2302 |
|
| 63 |
-
| 6.9011 | 1.29 | 160 | 7.1971 |
|
| 64 |
-
| 7.2194 | 1.45 | 180 | 7.0846 |
|
| 65 |
-
| 7.2281 | 1.61 | 200 | 7.0792 |
|
| 66 |
-
| 7.3379 | 1.77 | 220 | 7.0411 |
|
| 67 |
-
| 7.1461 | 1.94 | 240 | 7.0113 |
|
| 68 |
-
| 7.113 | 2.1 | 260 | 6.9626 |
|
| 69 |
-
| 6.7317 | 2.26 | 280 | 7.0031 |
|
| 70 |
-
| 6.8143 | 2.42 | 300 | 6.9219 |
|
| 71 |
-
| 6.9195 | 2.58 | 320 | 6.8336 |
|
| 72 |
-
| 6.3028 | 2.74 | 340 | 6.8596 |
|
| 73 |
-
| 6.8106 | 2.9 | 360 | 6.8300 |
|
| 74 |
-
|
| 75 |
-
|
| 76 |
### Framework versions
|
| 77 |
|
| 78 |
- Transformers 4.34.0.dev0
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
base_model: mistralai/Mistral-7B-v0.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
---
|
| 5 |
+
# Tensoic - Mistral-7B-v0.1-alpaca-2k-test
|
| 6 |
+
Our first fine tune of the Mistral 7B on the Alpaca-2k-test dataset. Feel free to play around!
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
### Training hyperparameters
|
| 9 |
|
|
|
|
| 22 |
- lr_scheduler_warmup_steps: 10
|
| 23 |
- num_epochs: 3
|
| 24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
### Framework versions
|
| 26 |
|
| 27 |
- Transformers 4.34.0.dev0
|