Tool Call Support?
Are you planning to support tool calls with this model? When running in ollama it says hf.co/bartowski/Mistral-Small-24B-Instruct-2501-GGUF:Q6_K does not support tools while the one pulled straight from ollama works fine with tool calls, though they don't have this equivalent quant yet on there. It is also fine if you are not planning to support tool calls, but it is a primary use I have for this model.
If you have pulled the offcial model from Ollama, you can then do ollama show mistral-small:24b --modelfile, save the modelfile, replace the FROM line with FROM hf.co/bartowski/Mistral-Small-24B-Instruct-2501-GGUF:Q6_K and then ollama create mistral-small-tools -f <location of the patched file>
I suspect the difference is in the template they have.
Also, can you just post the generated modelfile here or to pastebin, I don't want to pull the model to get it, and I need to do something similar with it
Looks like that solution worked, so the difference was in the template, like you suspected.
Can you post the modelfile here or to pastebin please?