bartowski

/

starchat2-15b-v0.1-GGUF

like 9

Hi,

I currently use llamacpp python for codellama and mistral and this is my demo code for prompt format. I want to know how to include starcoder model for the same. what is the format and stop token?

Reference snippet

if use_system_prompt:
    input_prompt = f"[INST] <<SYS>>\n{system_prompt}\n<</SYS>>\n\n "
else:
    input_prompt = f"[INST] "
for interaction in history:
    input_prompt = input_prompt + str(interaction[0]) + " [/INST] " + str(interaction[1]) + " </s><s> [INST] "

input_prompt = input_prompt + str(message) + " [/INST] "

output snippet below for stop token i currently use -

output = llm(
        input_prompt,
        temperature=Env.TEMPERATURE,
        top_p=Env.TOP_P,
        top_k=Env.TOP_K,
        repeat_penalty=Env.REPEAT_PENALTY,
        max_tokens=max_tokens_input,
        stop=[
            "<|prompter|>",
            "<|endoftext|>",
            "<|endoftext|> \n",
            "ASSISTANT:",
            "USER:",
            "SYSTEM:",
        ],
        stream=True,
    )