AWQ
#1
by
alegchenko
- opened
Hello! How do you make awq quant ? Do you use some branch from https://github.com/casper-hansen/AutoAWQ ?
Hi, Yes I added/modified the some files and code snippets to make autoawq capable of working with cohere architecture.
you can take help from this link to see the changes: https://github.com/casper-hansen/AutoAWQ/pull/457/files/9e523a84a4acc29221fc0b008d077087bd557139
alijawad07
changed discussion status to
closed
AutoAWQ has been updated, so you can directly update your installation and use directly.
alijawad07
changed discussion status to
open
alijawad07
changed discussion status to
closed
Thank you, your branch was usefull for me, i have quntized Aya23-35B in intresting for me colibration data, and have + 10-20% of metrics against llama3 8B / aya23-8B fp 16