Update README.md
Browse filesAdd chat template for tokenizer
https://huggingface.co/LLM360/K2-Chat/discussions/1#66586a040525ce56fabcc6fe
README.md
CHANGED
@@ -41,7 +41,21 @@ gen_tokens = model.generate(input_ids, do_sample=True, max_new_tokens=128)
|
|
41 |
print("-"*20 + "Output for model" + 20 * '-')
|
42 |
print(tokenizer.batch_decode(gen_tokens)[0])
|
43 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
44 |
|
|
|
|
|
|
|
|
|
|
|
|
|
45 |
## LLM360 Developer Suite
|
46 |
We provide step-by-step finetuning tutorials for tech enthusiasts, AI practitioners and academic or industry researchers [here](https://www.llm360.ai/developer.html).
|
47 |
|
|
|
41 |
print("-"*20 + "Output for model" + 20 * '-')
|
42 |
print(tokenizer.batch_decode(gen_tokens)[0])
|
43 |
```
|
44 |
+
Alternatively, you can construct the prompt by applying the chat template of tokenizer on input conversation:
|
45 |
+
```python
|
46 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
47 |
+
|
48 |
+
tokenizer = AutoTokenizer.from_pretrained("LLM360/K2-Chat")
|
49 |
+
model = AutoModelForCausalLM.from_pretrained("LLM360/K2-Chat")
|
50 |
+
|
51 |
+
messages = [{"role": "user", "content": "what is the highest mountain on earth?"}]
|
52 |
|
53 |
+
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
|
54 |
+
gen_tokens = model.generate(input_ids, do_sample=True, max_new_tokens=128)
|
55 |
+
|
56 |
+
print("-"*20 + "Output for model" + 20 * '-')
|
57 |
+
print(tokenizer.batch_decode(gen_tokens)[0])
|
58 |
+
```
|
59 |
## LLM360 Developer Suite
|
60 |
We provide step-by-step finetuning tutorials for tech enthusiasts, AI practitioners and academic or industry researchers [here](https://www.llm360.ai/developer.html).
|
61 |
|