Hugging Face's logo

bartowski
/
starchat2-15b-v0.1-GGUF

Text Generation
GGUF
alignment-handbook
Generated from Trainer
Inference Endpoints
conversational
Model card Files Community
2
bartowski commited on
Commit
f29a9ee
•
1 Parent(s): 7edc27c

Llamacpp quants

Browse files
Files changed (14) hide show
  1. .gitattributes +12 -0
  2. README.md +39 -0
  3. starchat2-15b-v0.1-Q2_K.gguf +3 -0
  4. starchat2-15b-v0.1-Q3_K_L.gguf +3 -0
  5. starchat2-15b-v0.1-Q3_K_M.gguf +3 -0
  6. starchat2-15b-v0.1-Q3_K_S.gguf +3 -0
  7. starchat2-15b-v0.1-Q4_0.gguf +3 -0
  8. starchat2-15b-v0.1-Q4_K_M.gguf +3 -0
  9. starchat2-15b-v0.1-Q4_K_S.gguf +3 -0
  10. starchat2-15b-v0.1-Q5_0.gguf +3 -0
  11. starchat2-15b-v0.1-Q5_K_M.gguf +3 -0
  12. starchat2-15b-v0.1-Q5_K_S.gguf +3 -0
  13. starchat2-15b-v0.1-Q6_K.gguf +3 -0
  14. starchat2-15b-v0.1-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ starchat2-15b-v0.1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ starchat2-15b-v0.1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ starchat2-15b-v0.1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ starchat2-15b-v0.1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ starchat2-15b-v0.1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ starchat2-15b-v0.1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ starchat2-15b-v0.1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ starchat2-15b-v0.1-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ starchat2-15b-v0.1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ starchat2-15b-v0.1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ starchat2-15b-v0.1-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ starchat2-15b-v0.1-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,39 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: HuggingFaceH4/starchat2-15b-sft-v0.1
3
+ tags:
4
+ - alignment-handbook
5
+ - generated_from_trainer
6
+ datasets:
7
+ - HuggingFaceH4/ultrafeedback_binarized
8
+ - HuggingFaceH4/orca_dpo_pairs
9
+ model-index:
10
+ - name: starchat2-15b-v0.1
11
+ results: []
12
+ quantized_by: bartowski
13
+ pipeline_tag: text-generation
14
+ ---
15
+
16
+ ## Llamacpp Quantizations of starchat2-15b-v0.1
17
+
18
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2405">b2405</a> for quantization.
19
+
20
+ Original model: https://huggingface.co/HuggingFaceH4/starchat2-15b-v0.1
21
+
22
+ Download a file (not the whole branch) from below:
23
+
24
+ | Filename | Quant type | File Size | Description |
25
+ | -------- | ---------- | --------- | ----------- |
26
+ | [starchat2-15b-v0.1-Q8_0.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q8_0.gguf) | Q8_0 | 16.96GB | Extremely high quality, generally unneeded but max available quant. |
27
+ | [starchat2-15b-v0.1-Q6_K.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q6_K.gguf) | Q6_K | 13.10GB | Very high quality, near perfect, *recommended*. |
28
+ | [starchat2-15b-v0.1-Q5_K_M.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q5_K_M.gguf) | Q5_K_M | 11.43GB | High quality, very usable. |
29
+ | [starchat2-15b-v0.1-Q5_K_S.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q5_K_S.gguf) | Q5_K_S | 11.02GB | High quality, very usable. |
30
+ | [starchat2-15b-v0.1-Q5_0.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q5_0.gguf) | Q5_0 | 11.02GB | High quality, older format, generally not recommended. |
31
+ | [starchat2-15b-v0.1-Q4_K_M.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q4_K_M.gguf) | Q4_K_M | 9.86GB | Good quality, similar to 4.25 bpw. |
32
+ | [starchat2-15b-v0.1-Q4_K_S.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q4_K_S.gguf) | Q4_K_S | 9.25GB | Slightly lower quality with small space savings. |
33
+ | [starchat2-15b-v0.1-Q4_0.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q4_0.gguf) | Q4_0 | 9.06GB | Decent quality, older format, generally not recommended. |
34
+ | [starchat2-15b-v0.1-Q3_K_L.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q3_K_L.gguf) | Q3_K_L | 8.96GB | Lower quality but usable, good for low RAM availability. |
35
+ | [starchat2-15b-v0.1-Q3_K_M.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q3_K_M.gguf) | Q3_K_M | 8.10GB | Even lower quality. |
36
+ | [starchat2-15b-v0.1-Q3_K_S.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q3_K_S.gguf) | Q3_K_S | 6.98GB | Low quality, not recommended. |
37
+ | [starchat2-15b-v0.1-Q2_K.gguf](https://huggingface.co/bartowski/starchat2-15b-v0.1-GGUF/blob/main/starchat2-15b-v0.1-Q2_K.gguf) | Q2_K | 6.19GB | Extremely low quality, *not* recommended.
38
+
39
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
starchat2-15b-v0.1-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd9153bdbbc316837647529d550fc1506ebfd499ee0385d1df39175d29826631
3
+ size 6192971680
starchat2-15b-v0.1-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9bf3b567cc4cd27167eb20cfb979cd5430399baa55f6c4917cf2196b0e39d1d3
3
+ size 8965342336
starchat2-15b-v0.1-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bcfcee81ea57d545bb4acfd71d4205eea8d67462299d93c15bc3318f6dbb1e1e
3
+ size 8101053568
starchat2-15b-v0.1-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a75f950ed686097993418ffb7e26f76a40fe8a6aee00c1afd161cca4f9a14b07
3
+ size 6986482816
starchat2-15b-v0.1-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7a49163e3bcbd4a52695a8f5f0527ab9aa657bddfc7a6f1770bbb0d5667ac79
3
+ size 9065417440
starchat2-15b-v0.1-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d79128c4ea9b382188ec9df22bb0c2eb3743a37ceaf97859a637cc53662d1cd
3
+ size 9860205280
starchat2-15b-v0.1-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5853a4f33a8eaafbdb82b7a7aa582d55b3fcb2ad394fe99dfc91b34158bff0c4
3
+ size 9255733984
starchat2-15b-v0.1-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04a4fe609f76e06cc684ae0252139269dc2e60e461bc29067a7930bf5302ef3e
3
+ size 11022061792
starchat2-15b-v0.1-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e838ed5e1f99e7ebad89c88389e500a981f713b23cb455dd0adcb5aad5a0e16
3
+ size 11431497952
starchat2-15b-v0.1-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b2bbe34f2419512a3ec655758bacf24ad9b693e0fb244be5e877daf8daf5bf4
3
+ size 11022061792
starchat2-15b-v0.1-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d2ec50bbaf4c279151974623178998fd27f8353ac61bef2f04468086d2065c4
3
+ size 13100996416
starchat2-15b-v0.1-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0791bcd074227be976ac30f5a6a7eb81c52d9522c5708b749e37d0f55e42b397
3
+ size 16965136000