IQ4_XS?

#1
by AaronFeng753 - opened

32B IQ4_XS + 32k context (Q8 KV cache) is perfect for 24gb cards

Thank you so much for uploading these ggufs!

Unsloth AI org

32B IQ4_XS + 32k context (Q8 KV cache) is perfect for 24gb cards

Thank you so much for uploading these ggufs!

We'll see what we can do! :)

Sign up or log in to comment