File size: 1,183 Bytes
9ddb2e6 d39bf60 9ddb2e6 1aba652 e12fd8f 1aba652 9ddb2e6 ce3134d cc4ce28 9ddb2e6 1aba652 ce3134d 1aba652 ce3134d a951fbe ce3134d a951fbe ce3134d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
---
base_model: unsloth/llama-3-8b-bnb-4bit
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
---
## Still in Beta!!!
This model will be used for the base model in J.O.S.I.E.v4o and furhter trained.
# Uploaded model
- **Developed by:** Isaak-Carter
- **License:** apache-2.0
- **Finetuned from model :** Isaak-Carter/JOSIEv4o-8b-stage1-beta1
- **Dataset used :** Isaak-Carter/j.o.s.i.e.v4.0.1o
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
```json
"per_device_train_batch_size": 2,
"gradient_accumulation_steps": 8,
"max_steps": 300,
"learning_rate": 2e-4,
"optim": "adamw_8bit",
"weight_decay": 0.01,
"lr_scheduler_type": "cosine"
```
## trained on this Prompt format
```txt
"""<|begin_of_text|>system
You are J.O.S.I.E. which is an acronym for "Just an Outstandingly Smart Intelligent Entity", a private and super-intelligent AI assistant, created by Gökdeniz Gülmez.<|end_of_text|>
<|begin_of_text|>main user "Gökdeniz Gülmez"
{{ .Prompt }}<|end_of_text|>
<|begin_of_text|>josie
{{ .Response }}<|end_of_text|>"""
|