README.md

---
license: unknown
---

[ehartford/WizardLM-7B-Uncensored](https://huggingface.co/ehartford/WizardLM-7B-Uncensored) quantized to **8bit GPTQ** with group size 128 + true sequential, no act order.

*For most uses this probably isn't what you want.* \
*For 4bit GPTQ quantizations see [TheBloke/WizardLM-7B-uncensored-GPTQ](https://huggingface.co/TheBloke/WizardLM-7B-uncensored-GPTQ)*

Quantized using AutoGPTQ with the following config:
```python
config: dict = dict(
    quantize_config=dict(model_file_base_name='WizardLM-7B-Uncensored',
                         bits=8, desc_act=False, group_size=128, true_sequential=True),
    use_safetensors=True
)
```
See `quantize.py` for the full script.

Tested for compatibility with:
- WSL with GPTQ-for-Llama `triton` branch.

AutoGPTQ loader should read configuration from `quantize_config.json`.\
For GPTQ-for-Llama use the following configuration when loading:\
wbits: 8\
groupsize: 128\
model_type: llama