Qwen
/

Qwen2.5-32B-Instruct-GPTQ-Int4

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

Resources

View closed (1)

Generating through HuggingFace Transformers leads to RuntimeError: probability tensor contains either `inf`, `nan` or element < 0. Generating through vLLM encounters no issues.

#2 opened 9 days ago by