Hugging Face's logo

sail
/
Sailor-4B-Chat-gguf

GGUF
multilingual
sea
sailor
sft
chat
instruction
Inference Endpoints
conversational
Model card Files Community
chaoscodes commited on
Commit
6ad2fb4
•
1 Parent(s): 4526779

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -65,17 +65,17 @@ Finally, we continually pre-train the Qwen1.5-0.5B model with 400 Billion tokens
65
  ### GGUF model list
66
  | Name | Quant method | Bits | Size | Use case |
67
  | ------------------------------------------------------------ | ------------ | ---- | ------- | ------------------------------------------------------------ |
68
- | [ggml-model-Q2_K.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q2_K.gguf) | Q2_K | 2 | 1.62 GB | smallest, significant quality loss ❗️ not recommended for most purposes |
69
- | [ggml-model-Q3_K_L.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q3_K_L.gguf) | Q3_K_L | 3 | 2.17 GB | small, substantial quality loss |
70
- | [ggml-model-Q3_K_M.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q3_K_M.gguf) | Q3_K_M | 3 | 2.03 GB | very small, balanced quality |
71
- | [ggml-model-Q3_K_S.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q3_K_S.gguf) | Q3_K_S | 3 | 1.86 GB | very small, high quality loss |
72
  | [ggml-model-Q4_K_M.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q4_K_M.gguf) | Q4_K_M | 4 | 2.46 GB | medium, balanced quality |
73
- | [ggml-model-Q4_K_S.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q4_K_S.gguf) | Q4_K_S | 4 | 2.34 GB | small, greater quality loss |
74
- | [ggml-model-Q5_K_M.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q5_K_M.gguf) | Q5_K_M | 5 | 2.84 GB | large, balanced quality |
75
  | [ggml-model-Q5_K_S.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q5_K_S.gguf) | Q5_K_S | 5 | 2.78 GB | medium, very low quality loss |
76
- | [ggml-model-Q6_K.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q6_K.gguf) | Q6_K | 6 | 3.25 GB | very large, extremely low quality loss |
77
- | [ggml-model-Q8_0.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q8_0.gguf) | Q8_0 | 8 | 4.2 GB | very large, extremely low quality loss |
78
- | [ggml-model-f16.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-f16.gguf) | f16 | 16 | 7.91 GB | original size, no quality loss |
79
 
80
  ### How to run with `llama.cpp`
81
 
 
65
  ### GGUF model list
66
  | Name | Quant method | Bits | Size | Use case |
67
  | ------------------------------------------------------------ | ------------ | ---- | ------- | ------------------------------------------------------------ |
68
+ | [ggml-model-Q2_K.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q2_K.gguf) | Q2_K | 2 | 1.62 GB | small, significant quality loss ❗️ not recommended for most purposes |
69
+ | [ggml-model-Q3_K_L.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q3_K_L.gguf) | Q3_K_L | 3 | 2.17 GB | medium, substantial quality loss |
70
+ | [ggml-model-Q3_K_M.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q3_K_M.gguf) | Q3_K_M | 3 | 2.03 GB | medium, balanced quality |
71
+ | [ggml-model-Q3_K_S.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q3_K_S.gguf) | Q3_K_S | 3 | 1.86 GB | small, high quality loss |
72
  | [ggml-model-Q4_K_M.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q4_K_M.gguf) | Q4_K_M | 4 | 2.46 GB | medium, balanced quality |
73
+ | [ggml-model-Q4_K_S.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q4_K_S.gguf) | Q4_K_S | 4 | 2.34 GB | medium, greater quality loss |
74
+ | [ggml-model-Q5_K_M.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q5_K_M.gguf) | Q5_K_M | 5 | 2.84 GB | medium, balanced quality |
75
  | [ggml-model-Q5_K_S.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q5_K_S.gguf) | Q5_K_S | 5 | 2.78 GB | medium, very low quality loss |
76
+ | [ggml-model-Q6_K.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q6_K.gguf) | Q6_K | 6 | 3.25 GB | medium, extremely low quality loss |
77
+ | [ggml-model-Q8_0.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-Q8_0.gguf) | Q8_0 | 8 | 4.2 GB | large, extremely low quality loss |
78
+ | [ggml-model-f16.gguf](https://huggingface.co/sail/Sailor-4B-Chat-gguf/blob/main/ggml-model-f16.gguf) | f16 | 16 | 7.91 GB | very large, no quality loss |
79
 
80
  ### How to run with `llama.cpp`
81