xiaotinghe/Qwen2.5-7B-Instruct-Wukong

Qwen2-0.5B-Instruct-Wukong

Requirements

The code of Qwen2 has been in the latest Hugging face transformers and we advise you to install transformers>=4.37.0, or you might encounter the following error:

KeyError: 'qwen2'

Quickstart

Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.

from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained(
    "xiaotinghe/Qwen2.5-7B-Instruct-Wukong",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("xiaotinghe/Qwen2.5-7B-Instruct-Wukong")

prompt = '以下是关于黑神话：悟空的单项选择题，请直接给出正确答案的选项。\n\n题目：百目真人的精魄属于哪种类型？\nA. 特品精魄\nB. 普通材料\nC. 普通精魄\nD. 稀有精魄\n答案：'
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]