Hugging Face's logo

tensorblock
/
Qwen2-7B-Instruct-GGUF

Transformers
GGUF
English
unsloth
qwen2
TensorBlock
GGUF
Inference Endpoints
conversational
Model card Files Community
morriszms commited on
Commit
6a798cb
•
1 Parent(s): ca9c12a

Upload folder using huggingface_hub

Browse files
Files changed (13) hide show
  1. Qwen2-7B-Instruct-Q2_K.gguf +2 -2
  2. Qwen2-7B-Instruct-Q3_K_L.gguf +2 -2
  3. Qwen2-7B-Instruct-Q3_K_M.gguf +2 -2
  4. Qwen2-7B-Instruct-Q3_K_S.gguf +2 -2
  5. Qwen2-7B-Instruct-Q4_0.gguf +2 -2
  6. Qwen2-7B-Instruct-Q4_K_M.gguf +2 -2
  7. Qwen2-7B-Instruct-Q4_K_S.gguf +2 -2
  8. Qwen2-7B-Instruct-Q5_0.gguf +2 -2
  9. Qwen2-7B-Instruct-Q5_K_M.gguf +2 -2
  10. Qwen2-7B-Instruct-Q5_K_S.gguf +2 -2
  11. Qwen2-7B-Instruct-Q6_K.gguf +2 -2
  12. Qwen2-7B-Instruct-Q8_0.gguf +2 -2
  13. README.md +21 -21
Qwen2-7B-Instruct-Q2_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4b12c479f7d0da6dc9dcc7353c0b33144400f6d9a247fb860631e797ea38cfb1
3
- size 3015938144
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c86ee669b5a6985f522dbc511a9d27b93b1c6ef7185e10f55217cd62f2102a95
3
+ size 3015937952
Qwen2-7B-Instruct-Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:03555c2fc24b7411974ee1d06e5ed41e58ae2b2871b5dc1d7fb4604a3ee5bd9e
3
- size 4088457312
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b8e773ce1f945047d93a1ea62cc58648ddd53d0880b4eac904b576273de8bfa7
3
+ size 4088457120
Qwen2-7B-Instruct-Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e9ee9f493d2b0e2a009cd1e28bb6b37f36deba9b77e660dcb0fbab588928b78c
3
- size 3808389216
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44bb917511893994d8f9db07b28b35c99643dd0bf9d48b8a87e323881c850494
3
+ size 3808389024
Qwen2-7B-Instruct-Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:aa767300284f187d3c459f3d99588012c89f097d4e96783e2ca3c958326fb0d7
3
- size 3492366432
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75a339db0e8352f6eb82115fd613b463e2166017659e3c459537fb9fcf50b902
3
+ size 3492366240
Qwen2-7B-Instruct-Q4_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f77a9c17211498c1af4fb55bcdb3a6efbc1f91a69280189b9e0c821fc0be909b
3
- size 4431388768
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5b50307561666d0f35bb826c2540f531dfaf6996743ddccccf561945d7d2f8dd
3
+ size 4431388576
Qwen2-7B-Instruct-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8f6f820efdc73b3b86f6078706f7963412ae199ea231351bc5a1611b1c520db3
3
- size 4683071584
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:578a5b2fe7ebf104b2a55ad315b9da3eff2085bef64a9e70ac07585417d22059
3
+ size 4683071392
Qwen2-7B-Instruct-Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dc574d94e48a174487ff4c7c8b3bc25c466b8295bf7cb5484ca2ce8b66ac198f
3
- size 4457767008
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c3c161b41d77082d3bc7f1f679629ac1540a4d47a6fa697c56ceb23350fb1ec3
3
+ size 4457766816
Qwen2-7B-Instruct-Q5_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:21768e601ca5683ed368adf22797ddd14f3555c06456fbc8310124767857f8df
3
- size 5315174496
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9768505a19981898b2efc957d327057cbb517430c080bd30c549ab4509844b8
3
+ size 5315174304
Qwen2-7B-Instruct-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3532e9839cc8773fc638397243e092146c8e5f755c18118c612c9c1068e37d5d
3
- size 5444829280
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5c70c7e476c0e8aae8c82dd23d4d699db8f923560e63d9247485e65d6bbd6843
3
+ size 5444829088
Qwen2-7B-Instruct-Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5141daab182609235490d0f4b8e70536b692d5504ecb25c069963c3b09bfa973
3
- size 5315174496
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0290801dd1cc5094e35f1cbefddb8332edcc3d688d242c4613c7a8f69704c839
3
+ size 5315174304
Qwen2-7B-Instruct-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3dd25810dab35eda04979f32894e5e34aefeb5e5e93378a4adc15d7d515818ea
3
- size 6254196832
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3aea292f4146b3f7128a854d3f68f52a745eff9445ac372d1b03baeef78eae5d
3
+ size 6254196640
Qwen2-7B-Instruct-Q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:13f84292290712722e1651acf94b597b461481bdefef732caa2b79cc0391d617
3
- size 8098523232
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4994c9aaa0d13357f215e09f8600a540179ba9e42cba184cf4cc3d7709d6cb8b
3
+ size 8098523040
README.md CHANGED
@@ -1,13 +1,15 @@
1
  ---
2
- license: apache-2.0
3
  language:
4
  - en
5
- pipeline_tag: text-generation
 
6
  tags:
7
- - chat
 
 
8
  - TensorBlock
9
  - GGUF
10
- base_model: Qwen/Qwen2-7B-Instruct
11
  ---
12
 
13
  <div style="width: auto; margin-left: auto; margin-right: auto">
@@ -21,12 +23,11 @@ base_model: Qwen/Qwen2-7B-Instruct
21
  </div>
22
  </div>
23
 
24
- ## Qwen/Qwen2-7B-Instruct - GGUF
25
 
26
- This repo contains GGUF format model files for [Qwen/Qwen2-7B-Instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct).
27
-
28
- The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
29
 
 
30
 
31
  <div style="text-align: left; margin: 20px 0;">
32
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
@@ -36,7 +37,6 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
36
 
37
  ## Prompt template
38
 
39
-
40
  ```
41
  <|im_start|>system
42
  {system_prompt}<|im_end|>
@@ -49,18 +49,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
49
 
50
  | Filename | Quant type | File Size | Description |
51
  | -------- | ---------- | --------- | ----------- |
52
- | [Qwen2-7B-Instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q2_K.gguf) | Q2_K | 2.809 GB | smallest, significant quality loss - not recommended for most purposes |
53
- | [Qwen2-7B-Instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q3_K_S.gguf) | Q3_K_S | 3.253 GB | very small, high quality loss |
54
- | [Qwen2-7B-Instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q3_K_M.gguf) | Q3_K_M | 3.547 GB | very small, high quality loss |
55
- | [Qwen2-7B-Instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q3_K_L.gguf) | Q3_K_L | 3.808 GB | small, substantial quality loss |
56
- | [Qwen2-7B-Instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q4_0.gguf) | Q4_0 | 4.127 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
57
- | [Qwen2-7B-Instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q4_K_S.gguf) | Q4_K_S | 4.152 GB | small, greater quality loss |
58
- | [Qwen2-7B-Instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q4_K_M.gguf) | Q4_K_M | 4.361 GB | medium, balanced quality - recommended |
59
- | [Qwen2-7B-Instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q5_0.gguf) | Q5_0 | 4.950 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
60
- | [Qwen2-7B-Instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q5_K_S.gguf) | Q5_K_S | 4.950 GB | large, low quality loss - recommended |
61
- | [Qwen2-7B-Instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q5_K_M.gguf) | Q5_K_M | 5.071 GB | large, very low quality loss - recommended |
62
- | [Qwen2-7B-Instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q6_K.gguf) | Q6_K | 5.825 GB | very large, extremely low quality loss |
63
- | [Qwen2-7B-Instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q8_0.gguf) | Q8_0 | 7.542 GB | very large, extremely low quality loss - not recommended |
64
 
65
 
66
  ## Downloading instruction
 
1
  ---
 
2
  language:
3
  - en
4
+ library_name: transformers
5
+ license: apache-2.0
6
  tags:
7
+ - unsloth
8
+ - transformers
9
+ - qwen2
10
  - TensorBlock
11
  - GGUF
12
+ base_model: unsloth/Qwen2-7B-Instruct
13
  ---
14
 
15
  <div style="width: auto; margin-left: auto; margin-right: auto">
 
23
  </div>
24
  </div>
25
 
26
+ ## unsloth/Qwen2-7B-Instruct - GGUF
27
 
28
+ This repo contains GGUF format model files for [unsloth/Qwen2-7B-Instruct](https://huggingface.co/unsloth/Qwen2-7B-Instruct).
 
 
29
 
30
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4242](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
31
 
32
  <div style="text-align: left; margin: 20px 0;">
33
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
 
37
 
38
  ## Prompt template
39
 
 
40
  ```
41
  <|im_start|>system
42
  {system_prompt}<|im_end|>
 
49
 
50
  | Filename | Quant type | File Size | Description |
51
  | -------- | ---------- | --------- | ----------- |
52
+ | [Qwen2-7B-Instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q2_K.gguf) | Q2_K | 3.016 GB | smallest, significant quality loss - not recommended for most purposes |
53
+ | [Qwen2-7B-Instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q3_K_S.gguf) | Q3_K_S | 3.492 GB | very small, high quality loss |
54
+ | [Qwen2-7B-Instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q3_K_M.gguf) | Q3_K_M | 3.808 GB | very small, high quality loss |
55
+ | [Qwen2-7B-Instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q3_K_L.gguf) | Q3_K_L | 4.088 GB | small, substantial quality loss |
56
+ | [Qwen2-7B-Instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q4_0.gguf) | Q4_0 | 4.431 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
57
+ | [Qwen2-7B-Instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q4_K_S.gguf) | Q4_K_S | 4.458 GB | small, greater quality loss |
58
+ | [Qwen2-7B-Instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q4_K_M.gguf) | Q4_K_M | 4.683 GB | medium, balanced quality - recommended |
59
+ | [Qwen2-7B-Instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q5_0.gguf) | Q5_0 | 5.315 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
60
+ | [Qwen2-7B-Instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q5_K_S.gguf) | Q5_K_S | 5.315 GB | large, low quality loss - recommended |
61
+ | [Qwen2-7B-Instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q5_K_M.gguf) | Q5_K_M | 5.445 GB | large, very low quality loss - recommended |
62
+ | [Qwen2-7B-Instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q6_K.gguf) | Q6_K | 6.254 GB | very large, extremely low quality loss |
63
+ | [Qwen2-7B-Instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Qwen2-7B-Instruct-GGUF/blob/main/Qwen2-7B-Instruct-Q8_0.gguf) | Q8_0 | 8.099 GB | very large, extremely low quality loss - not recommended |
64
 
65
 
66
  ## Downloading instruction