morriszms commited on
Commit
a4de5fc
1 Parent(s): 3d9825c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -12
README.md CHANGED
@@ -5500,8 +5500,16 @@ This repo contains GGUF format model files for [Alibaba-NLP/gte-Qwen2-7B-instruc
5500
 
5501
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
5502
 
 
 
 
 
 
 
 
5503
  ## Prompt template
5504
 
 
5505
  ```
5506
  <|im_start|>system
5507
  {system_prompt}<|im_end|>
@@ -5514,18 +5522,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
5514
 
5515
  | Filename | Quant type | File Size | Description |
5516
  | -------- | ---------- | --------- | ----------- |
5517
- | [gte-Qwen2-7B-instruct-Q2_K.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q2_K.gguf) | Q2_K | 2.807 GB | smallest, significant quality loss - not recommended for most purposes |
5518
- | [gte-Qwen2-7B-instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q3_K_S.gguf) | Q3_K_S | 3.251 GB | very small, high quality loss |
5519
- | [gte-Qwen2-7B-instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q3_K_M.gguf) | Q3_K_M | 3.545 GB | very small, high quality loss |
5520
- | [gte-Qwen2-7B-instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q3_K_L.gguf) | Q3_K_L | 3.806 GB | small, substantial quality loss |
5521
- | [gte-Qwen2-7B-instruct-Q4_0.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q4_0.gguf) | Q4_0 | 4.125 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
5522
- | [gte-Qwen2-7B-instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q4_K_S.gguf) | Q4_K_S | 4.150 GB | small, greater quality loss |
5523
- | [gte-Qwen2-7B-instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q4_K_M.gguf) | Q4_K_M | 4.360 GB | medium, balanced quality - recommended |
5524
- | [gte-Qwen2-7B-instruct-Q5_0.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q5_0.gguf) | Q5_0 | 4.948 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
5525
- | [gte-Qwen2-7B-instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q5_K_S.gguf) | Q5_K_S | 4.948 GB | large, low quality loss - recommended |
5526
- | [gte-Qwen2-7B-instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q5_K_M.gguf) | Q5_K_M | 5.069 GB | large, very low quality loss - recommended |
5527
- | [gte-Qwen2-7B-instruct-Q6_K.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q6_K.gguf) | Q6_K | 5.822 GB | very large, extremely low quality loss |
5528
- | [gte-Qwen2-7B-instruct-Q8_0.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/tree/main/gte-Qwen2-7B-instruct-Q8_0.gguf) | Q8_0 | 7.539 GB | very large, extremely low quality loss - not recommended |
5529
 
5530
 
5531
  ## Downloading instruction
 
5500
 
5501
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
5502
 
5503
+
5504
+ <div style="text-align: left; margin: 20px 0;">
5505
+ <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
5506
+ Run them on the TensorBlock client using your local machine ↗
5507
+ </a>
5508
+ </div>
5509
+
5510
  ## Prompt template
5511
 
5512
+
5513
  ```
5514
  <|im_start|>system
5515
  {system_prompt}<|im_end|>
 
5522
 
5523
  | Filename | Quant type | File Size | Description |
5524
  | -------- | ---------- | --------- | ----------- |
5525
+ | [gte-Qwen2-7B-instruct-Q2_K.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q2_K.gguf) | Q2_K | 2.807 GB | smallest, significant quality loss - not recommended for most purposes |
5526
+ | [gte-Qwen2-7B-instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q3_K_S.gguf) | Q3_K_S | 3.251 GB | very small, high quality loss |
5527
+ | [gte-Qwen2-7B-instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q3_K_M.gguf) | Q3_K_M | 3.545 GB | very small, high quality loss |
5528
+ | [gte-Qwen2-7B-instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q3_K_L.gguf) | Q3_K_L | 3.806 GB | small, substantial quality loss |
5529
+ | [gte-Qwen2-7B-instruct-Q4_0.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q4_0.gguf) | Q4_0 | 4.125 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
5530
+ | [gte-Qwen2-7B-instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q4_K_S.gguf) | Q4_K_S | 4.150 GB | small, greater quality loss |
5531
+ | [gte-Qwen2-7B-instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q4_K_M.gguf) | Q4_K_M | 4.360 GB | medium, balanced quality - recommended |
5532
+ | [gte-Qwen2-7B-instruct-Q5_0.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q5_0.gguf) | Q5_0 | 4.948 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
5533
+ | [gte-Qwen2-7B-instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q5_K_S.gguf) | Q5_K_S | 4.948 GB | large, low quality loss - recommended |
5534
+ | [gte-Qwen2-7B-instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q5_K_M.gguf) | Q5_K_M | 5.069 GB | large, very low quality loss - recommended |
5535
+ | [gte-Qwen2-7B-instruct-Q6_K.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q6_K.gguf) | Q6_K | 5.822 GB | very large, extremely low quality loss |
5536
+ | [gte-Qwen2-7B-instruct-Q8_0.gguf](https://huggingface.co/tensorblock/gte-Qwen2-7B-instruct-GGUF/blob/main/gte-Qwen2-7B-instruct-Q8_0.gguf) | Q8_0 | 7.539 GB | very large, extremely low quality loss - not recommended |
5537
 
5538
 
5539
  ## Downloading instruction