huihui-ai commited on
Commit
5ff6176
1 Parent(s): 969a8e2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -32,6 +32,33 @@ Qwen2.5-Coder uncensored version has covered six mainstream model sizes,
32
 
33
 
34
  If the desired result is not achieved, you can clear the conversation and try again.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35
  ## Usage
36
  You can use this model in your applications by loading it with Hugging Face's `transformers` library:
37
 
 
32
 
33
 
34
  If the desired result is not achieved, you can clear the conversation and try again.
35
+
36
+ ## ollama
37
+ 1. Download this model.
38
+ ```
39
+ huggingface-cli download huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated --local-dir ./huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
40
+ ```
41
+ 2. Get Qwen2.5-Coder-32B-Instruct model for reference.
42
+ ```
43
+ ollama pull qwen2.5-coder:32b
44
+ ```
45
+ 3. Export Qwen2.5-Coder-32B-Instruct model parameters.
46
+ ```
47
+ ollama show qwen2.5-coder:32b --modelfile > Modelfile
48
+ ```
49
+ 4. Modify Modelfile, Remove all comment lines (indicated by #) before the "FROM" keyword. Replace the "FROM" with the following content.
50
+ ```
51
+ FROM huihui-ai/Qwen2.5-Coder-32B-Instruct-abliterated
52
+ ```
53
+ 5. Use ollama create to then create the quantized model.
54
+ ```
55
+ ollama create --quantize q4_K_M -f Modelfile Qwen2.5-Coder-32B-Instruct-abliterated-q4_K_M
56
+ ```
57
+ 6. Run model
58
+ ```
59
+ ollama run Qwen2.5-Coder-32B-Instruct-abliterated-q4_K_M
60
+ ```
61
+
62
  ## Usage
63
  You can use this model in your applications by loading it with Hugging Face's `transformers` library:
64