Benchmarks worse than Qwen2.5-7B-Instruct on MMLU-Pro Computer Science in limited testing.

#1
by ubergarm - opened

Did a more detailed write-up here on r/localllama

Model MMLU-Pro Computer Science Score Generation tok/sec
bartowski/qwen2.5-7b-ins-v3-GGUF 51.71 104.76
bartowski/Qwen2.5-7B-Instruct-GGUF 59.51 170.51

Might be system prompt effecting it or something else, YMMV...

Sign up or log in to comment