Joseph717171
commited on
Commit
•
d8b5471
1
Parent(s):
4ad8cfb
Update README.md
Browse files
README.md
CHANGED
@@ -8,8 +8,13 @@ tags:
|
|
8 |
---
|
9 |
# Llama-3.1-8B-InitializedEmbeddings_with_Hermes-3
|
10 |
|
|
|
|
|
11 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
12 |
|
|
|
|
|
|
|
13 |
## Merge Details
|
14 |
### Merge Method
|
15 |
|
|
|
8 |
---
|
9 |
# Llama-3.1-8B-InitializedEmbeddings_with_Hermes-3
|
10 |
|
11 |
+
This is [Meta's LLaMA-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B) (base model) pre-initialized to [NousResearch/Hermes-3-Llama-3.1-8B's](https://huggingface.co/NousResearch/Hermes-3-Llama-3.1-8B) embedding's special tokens (prompt/chat template).
|
12 |
+
|
13 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
14 |
|
15 |
+
Credit for the idea of this merge goes to Charles Goddard (Creator of mergekit); his merge [chargoddard/Meta-Llama-3-8B-InitializedEmbeds](https://huggingface.co/chargoddard/Meta-Llama-3-8B-InitializedEmbeds) outlined the details and explained how it all worked, and why it is necessary to pre-initialize the base model with the instruct model's embedding's special tokens.
|
16 |
+
|
17 |
+
|
18 |
## Merge Details
|
19 |
### Merge Method
|
20 |
|