Owen Arliawan commited on
Commit
ac8a8ff
1 Parent(s): 6b6df25

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -1
README.md CHANGED
@@ -11,7 +11,7 @@ Trained on 2048 sequence length, while the base model is 8192 sequence length. F
11
 
12
  Trained using Cognitive Computations Eric Hartford's https://huggingface.co/datasets/cognitivecomputations/dolphin dataset as we've found great results from their dolphin models in previous Llama models.
13
 
14
- Trained for 2 days on 2x RTX3090 on our own machine, using 4-bit loading and Qlora 64-rank 128-alpha resulting in 2% trainable weights.
15
 
16
  The goal for this model is to have the model less-censored and great at general tasks like the previous dolphin models by Eric Hartford.
17
  We started training this BEFORE they launched their own full weight trained Llama-3-8B-Dolphin-2.9 with their own curated datasets and the newer "Dolphin 2.9" dataset.
@@ -31,6 +31,12 @@ Instruct format:
31
  {{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
32
  ```
33
 
 
 
 
 
 
 
34
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
35
 
36
  Axolotl Config:
 
11
 
12
  Trained using Cognitive Computations Eric Hartford's https://huggingface.co/datasets/cognitivecomputations/dolphin dataset as we've found great results from their dolphin models in previous Llama models.
13
 
14
+ Trained for 2 days on 2x RTX3090 on our own machine, using 4-bit loading and Qlora 64-rank 128-alpha resulting in ~2% trainable weights.
15
 
16
  The goal for this model is to have the model less-censored and great at general tasks like the previous dolphin models by Eric Hartford.
17
  We started training this BEFORE they launched their own full weight trained Llama-3-8B-Dolphin-2.9 with their own curated datasets and the newer "Dolphin 2.9" dataset.
 
31
  {{ user_message_2 }}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
32
  ```
33
 
34
+ Quants:
35
+
36
+ GGUF:
37
+
38
+ ExllamaV2:
39
+
40
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
41
 
42
  Axolotl Config: