GPTQ 8bit无法推理

by greatzane - opened Mar 29

Mar 29

•

我使用8bit的GPTQ会报如下错误（代码就是Model card页的例子代码）：

而使用4bit的没有问题。请问如何解决？

Apr 8

I am encountering the same issue...

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment