mylibrar commited on
Commit
66a39e6
1 Parent(s): aaf824f

Update README.md

Browse files

Add chat template for tokenizer
https://huggingface.co/LLM360/K2-Chat/discussions/1#66586a040525ce56fabcc6fe

Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -41,7 +41,21 @@ gen_tokens = model.generate(input_ids, do_sample=True, max_new_tokens=128)
41
  print("-"*20 + "Output for model" + 20 * '-')
42
  print(tokenizer.batch_decode(gen_tokens)[0])
43
  ```
 
 
 
 
 
 
 
 
44
 
 
 
 
 
 
 
45
  ## LLM360 Developer Suite
46
  We provide step-by-step finetuning tutorials for tech enthusiasts, AI practitioners and academic or industry researchers [here](https://www.llm360.ai/developer.html).
47
 
 
41
  print("-"*20 + "Output for model" + 20 * '-')
42
  print(tokenizer.batch_decode(gen_tokens)[0])
43
  ```
44
+ Alternatively, you can construct the prompt by applying the chat template of tokenizer on input conversation:
45
+ ```python
46
+ from transformers import AutoModelForCausalLM, AutoTokenizer
47
+
48
+ tokenizer = AutoTokenizer.from_pretrained("LLM360/K2-Chat")
49
+ model = AutoModelForCausalLM.from_pretrained("LLM360/K2-Chat")
50
+
51
+ messages = [{"role": "user", "content": "what is the highest mountain on earth?"}]
52
 
53
+ input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
54
+ gen_tokens = model.generate(input_ids, do_sample=True, max_new_tokens=128)
55
+
56
+ print("-"*20 + "Output for model" + 20 * '-')
57
+ print(tokenizer.batch_decode(gen_tokens)[0])
58
+ ```
59
  ## LLM360 Developer Suite
60
  We provide step-by-step finetuning tutorials for tech enthusiasts, AI practitioners and academic or industry researchers [here](https://www.llm360.ai/developer.html).
61