Build AutoGPTQ from source
Can you please provide some kind of instructions on how to do this? Or is there an existing build whl file for this branch we can use? I can't build it and the whole process of trying to build it is sort of insane. Any help would be great.
Failed to build auto-gptq
ERROR: Could not build wheels for auto-gptq, which is required to install pyproject.toml-based projects
I had issues with building too, this solved it: https://github.com/PanQiWei/AutoGPTQ/commit/17eb3d782999dc21168b8473a9b518a5cb2cb880
I rebased the Mistral-fix branch with the build fix.
I built the whl file and followed all the instructions correctly to update it in Oobagooba. However, it doesn't load and there are only errors. If somebody does figure out how to get this working using the custom branch of AutoGPTQ, it would be really great to provide some instructions that show how to load it in the webui and get it to work. I'd love to try this out.