RuntimeError: shape '[4, 226, 24576]' is invalid for input of size 9256960

#26
by linkai-dl - opened

Getting this error with 70B, but I can get both 13B and 7B work with GPTQ.

File /usr/local/lib/python3.10/site-packages/auto_gptq/nn_modules/qlinear_old.py:249, in QuantLinear.forward(self, x)
    246     weight = weight.reshape(weight.shape[0] * weight.shape[1], weight.shape[2])
    248     out = torch.matmul(x.half(), weight)
--> 249 out = out.half().reshape(out_shape)
    250 out = out + self.bias if self.bias is not None else out
    251 return out

RuntimeError: shape '[4, 226, 24576]' is invalid for input of size 9256960

Make sure you've updated to latest AutoGPTQ (currently 0.3.2) and have Transformers 4.31.0.

If you are already on the latest version, please double check the file downloaded OK.

wow what a speed! thanks, will try upgrading.

linkai-dl changed discussion status to closed

I got every package updated but got another error:

ValueError: not enough values to unpack (expected 3, got 2)

I am sure the files were correct.

linkai-dl changed discussion status to open

I also met this problem, transformer 4.31 didn't solve it

Sign up or log in to comment