TheBloke
/

Mixtral-8x7B-v0.1-GPTQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Resources

View closed (1)

The generative output is strange

#11 opened 3 months ago by

Speeds compared to llama_cpp_python?

#10 opened 10 months ago by

Unable to start TGI service for TheBloke/Mixtral-8x7B-v0.1-GPTQ with num_shard as 4

#9 opened 10 months ago by

What would be the minimal Sagemaker instance to deploy this model ?

#7 opened 11 months ago by

ValueError: Unsupported model type mixtral

#6 opened 11 months ago by

RuntimeError: shape '[32, 8]' is invalid for input of size 0

#5 opened 11 months ago by

Are you going to release mixtral-8x7B-v0.1-awq

#4 opened 11 months ago by

Running the model using "pip install auto-gptq" still results in "CUDA extension not installed"

#3 opened 11 months ago by

TypeError: mixtral isn't supported yet.

#2 opened 11 months ago by

Build AutoGPTQ from source

#1 opened 11 months ago by