4bit
/

WizardLM-13B-Uncensored-4bit-128g

Text Generation

Inference Endpoints

Model card Files Files and versions Community

WizardLM-13B-Uncensored-4bit-128g / README.md

camenduru's picture

thanks to ausboss ❤

81d47ba over 1 year ago

|

history blame contribute delete

242 Bytes

	quantized this [model](https://huggingface.co/ehartford/WizardLM-13B-Uncensored)

	CUDA_VISIBLE_DEVICES=0 python llama.py ehartford/WizardLM-13B-Uncensored c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors 4bit-128g.safetensors