gmonsoon commited on
Commit
f6340b9
1 Parent(s): c4de58e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -1
README.md CHANGED
@@ -9,4 +9,29 @@ language:
9
  - su
10
  ---
11
 
12
- This model is created by merging GoToCompany/llama3-8b-cpt-sahabatai-v1-instruct using passthrough method to stacking its layers, the result is the model parameters expanded to 11B.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  - su
10
  ---
11
 
12
+ This model is created by merging GoToCompany/llama3-8b-cpt-sahabatai-v1-instruct using passthrough method to stacking its layers, the result is the model parameters expanded to 11B.
13
+
14
+ ## 💻 Usage
15
+
16
+ ```python
17
+ !pip install -qU transformers accelerate
18
+
19
+ from transformers import AutoTokenizer
20
+ import transformers
21
+ import torch
22
+
23
+ model = "gmonsoon/SahabatAI-Llama-11B-Test"
24
+ messages = [{"role": "user", "content": "What is a large language model?"}]
25
+
26
+ tokenizer = AutoTokenizer.from_pretrained(model)
27
+ prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
28
+ pipeline = transformers.pipeline(
29
+ "text-generation",
30
+ model=model,
31
+ torch_dtype=torch.float16,
32
+ device_map="auto",
33
+ )
34
+
35
+ outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
36
+ print(outputs[0]["generated_text"])
37
+ ```