Configuration Parsing
Warning:
In config.json: "quantization_config.bits" must be an integer
QwQ-32B - EXL2 4.5bpw
This is a 4.5bpw EXL2 quant of Qwen/QwQ-32B
Details about the model can be found at the above model page.
Perplexity Scoring
Below are the perplexity scores for the EXL2 models. A lower score is better.
| Quant Level | Perplexity Score |
|---|---|
| 8.0 | 6.4393 |
| 7.0 | 6.4452 |
| 6.0 | 6.4693 |
| 5.0 | 6.4732 |
| 4.5 | 6.5417 |
| 4.0 | 6.6190 |
- Downloads last month
- -