Update README.md
Browse files
README.md
CHANGED
@@ -32,6 +32,33 @@ Qwen2.5-Coder uncensored version has covered six mainstream model sizes,
|
|
32 |
|
33 |
|
34 |
If the desired result is not achieved, you can clear the conversation and try again.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
35 |
## Usage
|
36 |
You can use this model in your applications by loading it with Hugging Face's `transformers` library:
|
37 |
|
|
|
32 |
|
33 |
|
34 |
If the desired result is not achieved, you can clear the conversation and try again.
|
35 |
+
|
36 |
+
## ollama
|
37 |
+
1. Download this model.
|
38 |
+
```
|
39 |
+
huggingface-cli download huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated --local-dir ./huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
|
40 |
+
```
|
41 |
+
2. Get Qwen2.5-Coder-32B-Instruct model for reference.
|
42 |
+
```
|
43 |
+
ollama pull qwen2.5-coder:32b
|
44 |
+
```
|
45 |
+
3. Export Qwen2.5-Coder-32B-Instruct model parameters.
|
46 |
+
```
|
47 |
+
ollama show qwen2.5-coder:32b --modelfile > Modelfile
|
48 |
+
```
|
49 |
+
4. Modify Modelfile, Remove all comment lines (indicated by #) before the "FROM" keyword. Replace the "FROM" with the following content.
|
50 |
+
```
|
51 |
+
FROM huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
|
52 |
+
```
|
53 |
+
5. Use ollama create to then create the quantized model.
|
54 |
+
```
|
55 |
+
ollama create --quantize q4_K_M -f Modelfile Qwen2.5-Coder-32B-Instruct-abliterated-q4_K_M
|
56 |
+
```
|
57 |
+
6. Run model
|
58 |
+
```
|
59 |
+
ollama run Qwen2.5-Coder-32B-Instruct-abliterated-q4_K_M
|
60 |
+
```
|
61 |
+
|
62 |
## Usage
|
63 |
You can use this model in your applications by loading it with Hugging Face's `transformers` library:
|
64 |
|