Quants?
#4
by
Heralax
- opened
I want to run this with Augmentoolkit. For local model usage it usually uses the aphrodite engine, which takes awq or gptq quants (I mean I could quant it myself using lcpp and run a server with that but that's slower).
Are there quants available somewhere?
Thanks 👍
Heralax
changed discussion status to
closed