I'm guessing from the name, this is a LoRA finetuned model, but is it 4 bit? Or is there a different model I need if I want to run a 32B param at home?Cheers,James
4 bits. It says q4_0
Β· Sign up or log in to comment