Hugging Face's logo

bartowski
/
blossom-v5-mistral-7b-GGUF

Text Generation
GGUF
Chinese
English
Inference Endpoints
Model card Files Community
bartowski commited on
Commit
db6550a
•
1 Parent(s): 3bd3811

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. README.md +42 -0
  3. blossom-v5-mistral-7b-IQ3_M.gguf +3 -0
  4. blossom-v5-mistral-7b-IQ3_S.gguf +3 -0
  5. blossom-v5-mistral-7b-IQ4_NL.gguf +3 -0
  6. blossom-v5-mistral-7b-IQ4_XS.gguf +3 -0
  7. blossom-v5-mistral-7b-Q2_K.gguf +3 -0
  8. blossom-v5-mistral-7b-Q3_K_L.gguf +3 -0
  9. blossom-v5-mistral-7b-Q3_K_M.gguf +3 -0
  10. blossom-v5-mistral-7b-Q3_K_S.gguf +3 -0
  11. blossom-v5-mistral-7b-Q4_0.gguf +3 -0
  12. blossom-v5-mistral-7b-Q4_K_M.gguf +3 -0
  13. blossom-v5-mistral-7b-Q4_K_S.gguf +3 -0
  14. blossom-v5-mistral-7b-Q5_0.gguf +3 -0
  15. blossom-v5-mistral-7b-Q5_K_M.gguf +3 -0
  16. blossom-v5-mistral-7b-Q5_K_S.gguf +3 -0
  17. blossom-v5-mistral-7b-Q6_K.gguf +3 -0
  18. blossom-v5-mistral-7b-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ blossom-v5-mistral-7b-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ blossom-v5-mistral-7b-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ blossom-v5-mistral-7b-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ blossom-v5-mistral-7b-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ blossom-v5-mistral-7b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ blossom-v5-mistral-7b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ blossom-v5-mistral-7b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ blossom-v5-mistral-7b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ blossom-v5-mistral-7b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ blossom-v5-mistral-7b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ blossom-v5-mistral-7b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ blossom-v5-mistral-7b-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ blossom-v5-mistral-7b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ blossom-v5-mistral-7b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ blossom-v5-mistral-7b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ blossom-v5-mistral-7b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - Azure99/blossom-chat-v3
5
+ - Azure99/blossom-math-v4
6
+ - Azure99/blossom-wizard-v3
7
+ - Azure99/blossom-orca-v3
8
+ language:
9
+ - zh
10
+ - en
11
+ quantized_by: bartowski
12
+ pipeline_tag: text-generation
13
+ ---
14
+
15
+ ## Llamacpp Quantizations of blossom-v5-mistral-7b
16
+
17
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
18
+
19
+ Original model: https://huggingface.co/Azure99/blossom-v5-mistral-7b
20
+
21
+ Download a file (not the whole branch) from below:
22
+
23
+ | Filename | Quant type | File Size | Description |
24
+ | -------- | ---------- | --------- | ----------- |
25
+ | [blossom-v5-mistral-7b-Q8_0.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
26
+ | [blossom-v5-mistral-7b-Q6_K.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
27
+ | [blossom-v5-mistral-7b-Q5_K_M.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
28
+ | [blossom-v5-mistral-7b-Q5_K_S.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
29
+ | [blossom-v5-mistral-7b-Q5_0.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
30
+ | [blossom-v5-mistral-7b-Q4_K_M.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, similar to 4.25 bpw. |
31
+ | [blossom-v5-mistral-7b-Q4_K_S.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
32
+ | [blossom-v5-mistral-7b-IQ4_NL.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-IQ4_NL.gguf) | IQ4_NL | 4.15GB | Good quality, similar to Q4_K_S, new method of quanting, |
33
+ | [blossom-v5-mistral-7b-IQ4_XS.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-IQ4_XS.gguf) | IQ4_XS | 3.94GB | Decent quality, new method with similar performance to Q4. |
34
+ | [blossom-v5-mistral-7b-Q4_0.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
35
+ | [blossom-v5-mistral-7b-IQ3_M.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-IQ3_M.gguf) | IQ3_M | 3.28GB | Medium-low quality, new method with decent performance. |
36
+ | [blossom-v5-mistral-7b-IQ3_S.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-IQ3_S.gguf) | IQ3_S | 3.18GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
37
+ | [blossom-v5-mistral-7b-Q3_K_L.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
38
+ | [blossom-v5-mistral-7b-Q3_K_M.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
39
+ | [blossom-v5-mistral-7b-Q3_K_S.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
40
+ | [blossom-v5-mistral-7b-Q2_K.gguf](https://huggingface.co/bartowski/blossom-v5-mistral-7b-GGUF/blob/main/blossom-v5-mistral-7b-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
41
+
42
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
blossom-v5-mistral-7b-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0acbfb062d1fb25841f7cdbd991dd5c2225dfcfcb8a0afe62c18f3b61cace21
3
+ size 3284891360
blossom-v5-mistral-7b-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:40009f7f3db0089d0ba776dadee68908adf84a6b27de6be3724685cfe6edf201
3
+ size 3182393056
blossom-v5-mistral-7b-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1804a987fca447d5befed3aad284199d7ea5c9e0ae3ac62b76a51bc3ae478001
3
+ size 4155053792
blossom-v5-mistral-7b-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6bd29701ccadc7427cdb4aed6d71a201a90733904894c40808d4d37c9b0342f
3
+ size 3944388320
blossom-v5-mistral-7b-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c16efb3b8ecbfeda6ab72ae12bd84e113bd89c207641e3ee49f1eaa643b9466a
3
+ size 2719241952
blossom-v5-mistral-7b-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d59a0730398611cdf59be8d9d1a3e68450ef753a46e2304e075f45eddba152b
3
+ size 3822024416
blossom-v5-mistral-7b-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33e98f0f980fd9023381abc0263a8a5a58d0405b769c77b736f5bac4bb60b0a9
3
+ size 3518985952
blossom-v5-mistral-7b-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:55fa99e9c6c76dd2960c27433b71a9e7983cf983cadcf986c6f1d00e0876f8aa
3
+ size 3164567264
blossom-v5-mistral-7b-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c36f21e2bbdc48d5d80051f480d7bf328e0d9bd94a3d0be3afef7eaa6b4b3f94
3
+ size 4108916448
blossom-v5-mistral-7b-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e23acb481733eb6ddb2265b781e20e44000da2aeb259444621c5be8a763c35d
3
+ size 4368439008
blossom-v5-mistral-7b-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6bc8e5f340649b38333e9e936bdfde8e05c99850efdeafaa28d458419bfad1a0
3
+ size 4140373728
blossom-v5-mistral-7b-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:765956a6b3b5504ab46bcc94a1f4a051f99601a299347f76a62dc2e4216e88f6
3
+ size 4997715680
blossom-v5-mistral-7b-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d56992389c5dd73f85b6534dc828ff351cd2d1ccac9de85f11b0921f81f0a5b1
3
+ size 5131409120
blossom-v5-mistral-7b-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee45b15e873a02afbfe4bd342c4d66dbb10a63b2102737d2718f43e2fb73ef3b
3
+ size 4997715680
blossom-v5-mistral-7b-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18ec0bac6ba48ddf3a31b5047e62a0a445bde9d02ebeee7aa184c6c3c3b81854
3
+ size 5942064864
blossom-v5-mistral-7b-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:35273fa0133cd73304288ac1b0322227be48287f890cf85b6f94eadd626ff9cb
3
+ size 7695857376