Hey, thanks for implementing support for this model. I'm getting pretty good numbers for in llama.cpp for the Q5 quant. The model's thinking is pretty concise compared to Kimi-K2.6 and it seems to pay attention to details in the middle better than the local instances of GLM-5.1 I've ran. I'm getting 300T/s for prompt processing and 11T/s for inference on a 768GB cpu-gpu hybrid setup. As this is a massive MoE, I believe the numbers here would be even better on ik_llama.cpp. I've already tried running this quant there, and it fails to load. Is there any chance you could open a PR to port the changes there?
Quite honestly, I don't run ik_llama.cpp myself but without porting over the conversion, dequant, and inference code I'm not surprised it fails to load. There's enough divergence from the forks as well that I'm not particularly keen on trying to do it myself either. The code is available for someone else to do so though.
Got it, seeing as the author of that fork ported the initial changes, perhaps he can do this as well. Again, great work with the model. Definitely an underrated gem that's SOTA for 'local' models.
