Satwik11

/

Phi-3.5-Mini-instruct-quantized-autoround-asym-2bit

like 0

GPTQ format has bug for asym kernel, which will cause large accuracy drop for small models and 2 bits. Recommend to use AutoRound Format