Hugging Face's logo

bartowski
/
mistral-orpo-capybara-7k-GGUF

Text Generation
GGUF
English
Eval Results
Inference Endpoints
conversational
Model card Files Community
bartowski commited on
Commit
908f003
•
1 Parent(s): 75f6f42

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. README.md +67 -0
  3. mistral-orpo-capybara-7k-IQ3_M.gguf +3 -0
  4. mistral-orpo-capybara-7k-IQ3_S.gguf +3 -0
  5. mistral-orpo-capybara-7k-IQ4_NL.gguf +3 -0
  6. mistral-orpo-capybara-7k-IQ4_XS.gguf +3 -0
  7. mistral-orpo-capybara-7k-Q2_K.gguf +3 -0
  8. mistral-orpo-capybara-7k-Q3_K_L.gguf +3 -0
  9. mistral-orpo-capybara-7k-Q3_K_M.gguf +3 -0
  10. mistral-orpo-capybara-7k-Q3_K_S.gguf +3 -0
  11. mistral-orpo-capybara-7k-Q4_0.gguf +3 -0
  12. mistral-orpo-capybara-7k-Q4_K_M.gguf +3 -0
  13. mistral-orpo-capybara-7k-Q4_K_S.gguf +3 -0
  14. mistral-orpo-capybara-7k-Q5_0.gguf +3 -0
  15. mistral-orpo-capybara-7k-Q5_K_M.gguf +3 -0
  16. mistral-orpo-capybara-7k-Q5_K_S.gguf +3 -0
  17. mistral-orpo-capybara-7k-Q6_K.gguf +3 -0
  18. mistral-orpo-capybara-7k-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ mistral-orpo-capybara-7k-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ mistral-orpo-capybara-7k-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ mistral-orpo-capybara-7k-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ mistral-orpo-capybara-7k-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ mistral-orpo-capybara-7k-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ mistral-orpo-capybara-7k-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ mistral-orpo-capybara-7k-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ mistral-orpo-capybara-7k-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ mistral-orpo-capybara-7k-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ mistral-orpo-capybara-7k-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ mistral-orpo-capybara-7k-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ mistral-orpo-capybara-7k-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ mistral-orpo-capybara-7k-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ mistral-orpo-capybara-7k-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ mistral-orpo-capybara-7k-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ mistral-orpo-capybara-7k-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: mit
5
+ base_model:
6
+ - mistralai/Mistral-7B-v0.1
7
+ datasets:
8
+ - argilla/distilabel-capybara-dpo-7k-binarized
9
+ pipeline_tag: text-generation
10
+ model-index:
11
+ - name: Mistral-ORPO-Capybara-7k
12
+ results:
13
+ - task:
14
+ type: text-generation
15
+ dataset:
16
+ name: AlpacaEval 2 (LC)
17
+ type: AlpacaEval
18
+ metrics:
19
+ - type: AlpacaEval 2.0
20
+ value: 15.88%
21
+ name: Win Rate
22
+ source:
23
+ url: https://tatsu-lab.github.io/alpaca_eval/
24
+ name: self-reported
25
+ - task:
26
+ type: text-generation
27
+ dataset:
28
+ name: MT-Bench
29
+ type: MT-Bench
30
+ metrics:
31
+ - type: MT-Bench
32
+ value: 7.444
33
+ name: Score
34
+ source:
35
+ url: https://github.com/lm-sys/FastChat/blob/main/fastchat/llm_judge/
36
+ name: self-reported
37
+ quantized_by: bartowski
38
+ ---
39
+
40
+ ## Llamacpp Quantizations of mistral-orpo-capybara-7k
41
+
42
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
43
+
44
+ Original model: https://huggingface.co/kaist-ai/mistral-orpo-capybara-7k
45
+
46
+ Download a file (not the whole branch) from below:
47
+
48
+ | Filename | Quant type | File Size | Description |
49
+ | -------- | ---------- | --------- | ----------- |
50
+ | [mistral-orpo-capybara-7k-Q8_0.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
51
+ | [mistral-orpo-capybara-7k-Q6_K.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
52
+ | [mistral-orpo-capybara-7k-Q5_K_M.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
53
+ | [mistral-orpo-capybara-7k-Q5_K_S.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
54
+ | [mistral-orpo-capybara-7k-Q5_0.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
55
+ | [mistral-orpo-capybara-7k-Q4_K_M.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, similar to 4.25 bpw. |
56
+ | [mistral-orpo-capybara-7k-Q4_K_S.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
57
+ | [mistral-orpo-capybara-7k-IQ4_NL.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-IQ4_NL.gguf) | IQ4_NL | 4.15GB | Good quality, similar to Q4_K_S, new method of quanting, |
58
+ | [mistral-orpo-capybara-7k-IQ4_XS.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-IQ4_XS.gguf) | IQ4_XS | 3.94GB | Decent quality, new method with similar performance to Q4. |
59
+ | [mistral-orpo-capybara-7k-Q4_0.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
60
+ | [mistral-orpo-capybara-7k-IQ3_M.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-IQ3_M.gguf) | IQ3_M | 3.28GB | Medium-low quality, new method with decent performance. |
61
+ | [mistral-orpo-capybara-7k-IQ3_S.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-IQ3_S.gguf) | IQ3_S | 3.18GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
62
+ | [mistral-orpo-capybara-7k-Q3_K_L.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
63
+ | [mistral-orpo-capybara-7k-Q3_K_M.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
64
+ | [mistral-orpo-capybara-7k-Q3_K_S.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
65
+ | [mistral-orpo-capybara-7k-Q2_K.gguf](https://huggingface.co/bartowski/mistral-orpo-capybara-7k-GGUF/blob/main/mistral-orpo-capybara-7k-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
66
+
67
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
mistral-orpo-capybara-7k-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b569c5e17f600a01796365bcaf49df181b5628e1b4e2a14b28fa4749ff578e0e
3
+ size 3284891808
mistral-orpo-capybara-7k-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:652150c3a962e1249edec9d018488cea3493b0592aecf0bd46e904707814af3c
3
+ size 3182393504
mistral-orpo-capybara-7k-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:336a0dee4d442bf206030bc9514a230d3c06947c778f63ba5ec02f604a351519
3
+ size 4155054240
mistral-orpo-capybara-7k-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:472087fffe1277d50fb795b9ed455c6884088a623a132fc98fa955a52f16a51a
3
+ size 3944388768
mistral-orpo-capybara-7k-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd92ef5a59bed5a820b3a82ba3ea68b7937437d9e5270be51c57bbaa8c0a3e85
3
+ size 2719242400
mistral-orpo-capybara-7k-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d572b1d49f8faba081537cc45315ae06ea8b720e64ec75263e3e7d0cdd5b93b0
3
+ size 3822024864
mistral-orpo-capybara-7k-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d014ecb051bd590e798a45d588af806cbc12e1d8aac66ba4f09b84c59a14df1
3
+ size 3518986400
mistral-orpo-capybara-7k-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1122c9e9a82a06ea0d74d583ed54f4167f2f2242d3dfa7c7c1b59124391f4924
3
+ size 3164567712
mistral-orpo-capybara-7k-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db890bfaaef29f30a4fc8f6553a8f9ff277b435cad9a6e05cb44160cd2420a15
3
+ size 4108916896
mistral-orpo-capybara-7k-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04498db9824bcc59f51a0b336fe827b666cc0b5b9f36f933ecac6d73efa58cee
3
+ size 4368439456
mistral-orpo-capybara-7k-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f098b7051a3b0f8cc1f51f3c251049de98ac8e3c8b93590c778aa26381f0c364
3
+ size 4140374176
mistral-orpo-capybara-7k-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78b86671acd03ead597ea36fa5a6ebb139a188fb2e9384a15e149f8980b41339
3
+ size 4997716128
mistral-orpo-capybara-7k-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1f05cf11cf63f3626e7496be2962c50dd3ca711beadaf46b09360dde53aae7d8
3
+ size 5131409568
mistral-orpo-capybara-7k-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85a454e5c20ecfbb7003c6ce224d3bf4c5f49a3c1e789b5e8a294589fb2b53a5
3
+ size 4997716128
mistral-orpo-capybara-7k-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95b9ed5e65f7e3e4e6bda38cbcf793fab4d0fb54849611a9f4d246d991909b07
3
+ size 5942065312
mistral-orpo-capybara-7k-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cf00e823f897486b2f03d152fbd880bd2b3a905d0c93f226e6719d772d3a9d09
3
+ size 7695857824