metadata
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct-GGUF/blob/main/LICENSE
language:
- en
base_model:
- Qwen/Qwen2.5-Coder-32B-Instruct
pipeline_tag: text-generation
library_name: gguf
tags:
- code
- codeqwen
- chat
- qwen
- qwen-coder
Speculative decoding with Qwen 32B + Qwen 1.5B
Example:
llama-server \
-m qwen2.5-coder-32b-instruct-q4_k_m.gguf \
-md qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
--ctx-size 65536