Qwen
/

Qwen2.5-Coder-32B-Instruct

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

HQQ quants

#27

by wrtn2 - opened 6 days ago

wrtn2

6 days ago

Would be interesting to have HQQ quants, including 2bit, which in some cases has surprisingly low degradation. Is anyone aware of any? For science.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment