Is this model 4 bit?

#6
by JamesLear - opened

I'm guessing from the name, this is a LoRA finetuned model, but is it 4 bit? Or is there a different model I need if I want to run a 32B param at home?
Cheers,
James

4 bits. It says q4_0

Sign up or log in to comment