EXL2 quants of Qwen2-VL-72B-Instruct
4.00 bits per weight
4.50 bits per weight
5.00 bits per weight
6.00 bits per weight
(2.3bpw to 3.5bpw revisions are in also this repo, but they are unstable. Working on it.)
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.