General discussion.
@Lewdiculous lmao its in the que for the leaderboard, but ive been testing locally and it cooks so far. Waiting to stretch the context after a few more ideas i want to get inside the model.
Curious for leaderboard results 👀
improvement but not much:
Hey, it's an improvement nonetheless! 👏
Prima-LelantaclesV6-7b-Q4_K_M-imatrix.gguf
Prima-LelantaclesV6-7b-Q4_K_S-imatrix.gguf
Prima-LelantaclesV6-7b-IQ4_NL-imatrix.gguf
Prima-LelantaclesV6-7b-IQ4_XS-imatrix.gguf (only supported in koboldcpp 1.60 or higher)
Prima-LelantaclesV6-7b-Q5_K_M-imatrix.gguf
Prima-LelantaclesV6-7b-Q5_K_S-imatrix.gguf
Prima-LelantaclesV6-7b-Q6_K-imatrix.gguf
Prima-LelantaclesV6-7b-Q8_0-imatrix.gguf
Prima-LelantaclesV6-7b-IQ3_M-imatrix.gguf
Prima-LelantaclesV6-7b-IQ3_S-imatrix.gguf
Prima-LelantaclesV6-7b-IQ3_XS-imatrix.gguf
Prima-LelantaclesV6-7b-IQ3_XXS-imatrix.gguf
The GGUF situation is getting intense XD
I think things are sort of getting re-unified again soon, maybe? Hopefully.
Lewdiculous/Prima-LelantaclesV6-7b-GGUF-IQ-Imatrix
Fixed: Link
Appreciated as always! i went kind of nuts tonight and did a bunch of stuff. Will update with more information towards the weekend and as i have more time.
@Lewdiculous Is support coming in the next koboldcpp release or are you referring to 1.59.1? I cant find it in lostruins' releases
@saishf - Yes, I meant in the next release of KCPP, for the IQ4_XS support to be merged, this is the PR that needs to be merged downstream.
Okay, thank you!
The Chaotic Neutrals have cooked up a merge using the two front runner models from the org: Eris-7b coming soon. (LelantaclesV6 x Prodigy) from both jeiku and myself. https://huggingface.co/ChaoticNeutrals/Eris_7B
Great news. 🎉