File size: 1,153 Bytes
9ddb2e6
 
 
 
 
 
 
 
 
 
 
d39bf60
9ddb2e6
 
1aba652
e12fd8f
1aba652
9ddb2e6
 
 
 
ce3134d
cc4ce28
9ddb2e6
 
 
1aba652
 
 
ce3134d
1aba652
 
 
 
 
ce3134d
 
 
 
 
 
 
95a0d1a
ce3134d
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
---
base_model: unsloth/llama-3-8b-bnb-4bit
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
---

## Still in Beta!!!
This model will be used for the base model in J.O.S.I.E.v4o and furhter trained.

# Uploaded  model

- **Developed by:** Isaak-Carter
- **License:** apache-2.0
- **Finetuned from model :** Isaak-Carter/JOSIEv4o-8b-stage1-beta1
- **Dataset used :** Isaak-Carter/j.o.s.i.e.v4.0.1o

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

```json
  "per_device_train_batch_size": 2,
  "gradient_accumulation_steps": 8,
  "max_steps": 300,
  "learning_rate": 2e-4,
  "optim": "adamw_8bit",
  "weight_decay": 0.01,
  "lr_scheduler_type": "cosine"
```

## trained on this Prompt format

```txt
"""<|begin_of_text|>system
You are J.O.S.I.E. which is an acronym for "Just an Outstandingly Smart Intelligent Entity", a private and super-intelligent AI assistant, created by Gökdeniz Gülmez.
<|begin_of_text|>main user "Gökdeniz Gülmez"
{{ .Prompt }}
<|begin_of_text|>josie
{{ .Response }}<|end_of_text|>"""