Joseph717171 commited on
Commit
d8b5471
1 Parent(s): 4ad8cfb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -0
README.md CHANGED
@@ -8,8 +8,13 @@ tags:
8
  ---
9
  # Llama-3.1-8B-InitializedEmbeddings_with_Hermes-3
10
 
 
 
11
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
 
 
 
 
13
  ## Merge Details
14
  ### Merge Method
15
 
 
8
  ---
9
  # Llama-3.1-8B-InitializedEmbeddings_with_Hermes-3
10
 
11
+ This is [Meta's LLaMA-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B) (base model) pre-initialized to [NousResearch/Hermes-3-Llama-3.1-8B's](https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B) embedding's special tokens (prompt/chat template).
12
+
13
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
 
15
+ Credit for the idea of this merge goes to Charles Goddard (Creator of mergekit); his merge [chargoddard/Meta-Llama-3-8B-InitializedEmbeds](https://huggingface.co/chargoddard/Meta-Llama-3-8B-InitializedEmbeds) outlined the details and explained how it all worked, and why it is necessary to pre-initialize the base model with the instruct model's embedding's special tokens.
16
+
17
+
18
  ## Merge Details
19
  ### Merge Method
20