| This repo contains my preferred SillyTavern settings for Mistral Small models. | |
| Mistral Small is a fantastic model, but its user/assistant turn structure means that the context/prompt composed by SillyTavern can mess up the instruct formatting for things like Lorebook/World Info injection or Author's Note injection, or result in repeated Assistant or User turns that can confuse the model. | |
| I've had the best luck composing the context as a **single user turn**, asking the model to respond to what's essentially a very long single-turn prompt. | |
| The composed context tends to look something like this: | |
| ``` | |
| <s>[INST] System prompt: Use the following information to write {{char}}'s next response in the Exchange below. | |
| Character description | |
| Persona description | |
| Scenario description | |
| World Info | |
| Exchange: | |
| {{char}}: First message</s> | |
| {{user}}: First response | |
| {{char}}: Second message</s> | |
| {{user}}: Second response | |
| Author's Note: Write 2 paragraphs using vivid and natural prose.[/INST] {{char}}: | |
| ``` | |
| This prevents instruct formatting issues from AN/WI insertions, and avoids problems with repeated turns depending on whether the first message in the chat is from the Character or the User. | |
| When using AN with this template, it should work correctly with Before/After Story String insertion. For more immediate insertion, Depth 1 (as shown in the example above) is least likely to confuse the model as it doesn't interrupt the chat history. | |
| Note: `</s>` is added after each Character message in the chat history to prevent problems with the AI not ending its turn properly when writing the newest response. |