Spaces:
Running
on
Zero
Custom Format for Mistral
When i saw the llama-cpp-agent, the mistral prompt makers is looks like combining v1 and tekken chat template.
Based on Mistral Cookbook and inflatebot's SillyTavern template
i generate this code with LLM
from llama_cpp_agent.messages_formatter import MessagesFormatter, PromptMarkers, Roles
mistral_v1_markers = {
Roles.system: PromptMarkers(""" [INST] """, """ [/INST] Understood.</s>"""),
Roles.user: PromptMarkers(""" [INST] """, """ [/INST]"""),
Roles.assistant: PromptMarkers(" ", "</s>"),
Roles.tool: PromptMarkers("", ""),
}
mistral_v2_markers = {
Roles.system: PromptMarkers("""[INST] """, """[/INST] Understood.</s>"""),
Roles.user: PromptMarkers("""[INST] """, """[/INST]"""),
Roles.assistant: PromptMarkers(" ", "</s>"),
Roles.tool: PromptMarkers("", ""),
}
mistral_v3_tekken_markers = {
Roles.system: PromptMarkers("""[INST]""", """[/INST]Understood.</s>"""),
Roles.user: PromptMarkers("""[INST]""", """[/INST]"""),
Roles.assistant: PromptMarkers("", "</s>"),
Roles.tool: PromptMarkers("", ""),
}
mistral_v1_formatter = MessagesFormatter(
pre_prompt="",
prompt_markers=mistral_v1_markers,
include_sys_prompt_in_first_user_message=False,
default_stop_sequences=["</s>"]
)
mistral_v2_formatter = MessagesFormatter(
pre_prompt="",
prompt_markers=mistral_v2_markers,
include_sys_prompt_in_first_user_message=False,
default_stop_sequences=["</s>"]
)
mistral_v3_tekken_formatter = MessagesFormatter(
pre_prompt="",
prompt_markers=mistral_v3_tekken_markers,
include_sys_prompt_in_first_user_message=False,
default_stop_sequences=["</s>"]
)
i haven't tested it but maybe it's useful
Thanks for the contribution! I made a bit of a mistake at first, but your part of the code seems to work just fine as is. This change also makes it easier to add templates in the future.
I read the code on github when I bugged it, and it seems that the preset templates are not that complex to define. We can customize it without worry.
https://github.com/Maximilian-Winter/llama-cpp-agent/blob/master/src/llama_cpp_agent/messages_formatter.py
Glad to hear that.
Well, I think it's also great if it can be moved upstream
I'll have to get a github account first!
Well, the acquisition itself may be easy, but I can't take the time to get introduced to github.🥲
The modification of the github code itself is easy, the rest of the work is just to decide on the official name. The capitalized one.
Nice work