Upload folder using huggingface_hub

README.md

@@ -5,14 +5,17 @@ tags:
 - mergekit
 - lazymergekit
 - mistralai/Mistral-7B-Instruct-v0.2
-- janai-hq/trinity-v1
+- jan-hq/trinity-v1
+base_model:
+- mistralai/Mistral-7B-Instruct-v0.2
+- jan-hq/trinity-v1
 ---
 
 # MistralTrinity-7B-slerp
 
-MistralTrinity-7B-slerp is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
+MistralTrinity-7B-slerp is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
 * [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
-* [janai-hq/trinity-v1](https://huggingface.co/janai-hq/trinity-v1)
+* [jan-hq/trinity-v1](https://huggingface.co/jan-hq/trinity-v1)
 
 ## 🧩 Configuration
 
@@ -21,7 +24,7 @@ slices:
   - sources:
       - model: mistralai/Mistral-7B-Instruct-v0.2
         layer_range: [0, 32]
-      - model: janai-hq/trinity-v1
+      - model: jan-hq/trinity-v1
         layer_range: [0, 32]
 merge_method: slerp
 base_model: mistralai/Mistral-7B-Instruct-v0.2
@@ -33,5 +36,29 @@ parameters:
       value: [1, 0.5, 0.7, 0.3, 0]
     - value: 0.5
 dtype: bfloat16
+```
+
+## 💻 Usage
+
+```python
+!pip install -qU transformers accelerate
+
+from transformers import AutoTokenizer
+import transformers
+import torch
+
+model = "wenqiglantz/MistralTrinity-7B-slerp"
+messages = [{"role": "user", "content": "What is a large language model?"}]
+
+tokenizer = AutoTokenizer.from_pretrained(model)
+prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+pipeline = transformers.pipeline(
+    "text-generation",
+    model=model,
+    torch_dtype=torch.float16,
+    device_map="auto",
+)
 
+outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
+print(outputs[0]["generated_text"])
 ```

mergekit_config.yml

@@ -3,7 +3,7 @@ slices:
   - sources:
       - model: mistralai/Mistral-7B-Instruct-v0.2
         layer_range: [0, 32]
-      - model: janai-hq/trinity-v1
+      - model: jan-hq/trinity-v1
         layer_range: [0, 32]
 merge_method: slerp
 base_model: mistralai/Mistral-7B-Instruct-v0.2
@@ -15,4 +15,3 @@ parameters:
       value: [1, 0.5, 0.7, 0.3, 0]
     - value: 0.5
 dtype: bfloat16
-