bartowski commited on
Commit
290bec0
β€’
1 Parent(s): 1255509

Rename files to match mlabonne

Browse files
Files changed (21) hide show
  1. Llama-3.1-70B-Instruct-abliterated-IQ2_M.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ2_M.gguf +0 -0
  2. Llama-3.1-70B-Instruct-abliterated-IQ2_XS.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ2_XS.gguf +0 -0
  3. Llama-3.1-70B-Instruct-abliterated-IQ2_XXS.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ2_XXS.gguf +0 -0
  4. Llama-3.1-70B-Instruct-abliterated-IQ3_M.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ3_M.gguf +0 -0
  5. Llama-3.1-70B-Instruct-abliterated-IQ3_XXS.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ3_XXS.gguf +0 -0
  6. Llama-3.1-70B-Instruct-abliterated-IQ4_XS.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ4_XS.gguf +0 -0
  7. Llama-3.1-70B-Instruct-abliterated-Q2_K.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q2_K.gguf +0 -0
  8. Llama-3.1-70B-Instruct-abliterated-Q2_K_L.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q2_K_L.gguf +0 -0
  9. Llama-3.1-70B-Instruct-abliterated-Q3_K_L.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q3_K_L.gguf +0 -0
  10. Llama-3.1-70B-Instruct-abliterated-Q3_K_M.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q3_K_M.gguf +0 -0
  11. Llama-3.1-70B-Instruct-abliterated-Q3_K_XL.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q3_K_XL.gguf +0 -0
  12. Llama-3.1-70B-Instruct-abliterated-Q4_K_L.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q4_K_L.gguf +0 -0
  13. Llama-3.1-70B-Instruct-abliterated-Q4_K_M.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf +0 -0
  14. Llama-3.1-70B-Instruct-abliterated-Q5_K_M/Llama-3.1-70B-Instruct-abliterated-Q5_K_M-00001-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q5_K_M/Llama-3.1-70B-Instruct-lorablated-Q5_K_M-00001-of-00002.gguf +0 -0
  15. Llama-3.1-70B-Instruct-abliterated-Q5_K_M/Llama-3.1-70B-Instruct-abliterated-Q5_K_M-00002-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q5_K_M/Llama-3.1-70B-Instruct-lorablated-Q5_K_M-00002-of-00002.gguf +0 -0
  16. Llama-3.1-70B-Instruct-abliterated-Q6_K/Llama-3.1-70B-Instruct-abliterated-Q6_K-00001-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q6_K/Llama-3.1-70B-Instruct-lorablated-Q6_K-00001-of-00002.gguf +0 -0
  17. Llama-3.1-70B-Instruct-abliterated-Q6_K/Llama-3.1-70B-Instruct-abliterated-Q6_K-00002-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q6_K/Llama-3.1-70B-Instruct-lorablated-Q6_K-00002-of-00002.gguf +0 -0
  18. Llama-3.1-70B-Instruct-abliterated-Q8_0/Llama-3.1-70B-Instruct-abliterated-Q8_0-00001-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q8_0/Llama-3.1-70B-Instruct-lorablated-Q8_0-00001-of-00002.gguf +0 -0
  19. Llama-3.1-70B-Instruct-abliterated-Q8_0/Llama-3.1-70B-Instruct-abliterated-Q8_0-00002-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q8_0/Llama-3.1-70B-Instruct-lorablated-Q8_0-00002-of-00002.gguf +0 -0
  20. Llama-3.1-70B-Instruct-abliterated.imatrix β†’ Llama-3.1-70B-Instruct-lorablated.imatrix +0 -0
  21. README.md +26 -23
Llama-3.1-70B-Instruct-abliterated-IQ2_M.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ2_M.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-IQ2_XS.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ2_XS.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-IQ2_XXS.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ2_XXS.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-IQ3_M.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ3_M.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-IQ3_XXS.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ3_XXS.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-IQ4_XS.gguf β†’ Llama-3.1-70B-Instruct-lorablated-IQ4_XS.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q2_K.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q2_K.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q2_K_L.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q2_K_L.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q3_K_L.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q3_K_L.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q3_K_M.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q3_K_M.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q3_K_XL.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q3_K_XL.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q4_K_L.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q4_K_L.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q4_K_M.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q5_K_M/Llama-3.1-70B-Instruct-abliterated-Q5_K_M-00001-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q5_K_M/Llama-3.1-70B-Instruct-lorablated-Q5_K_M-00001-of-00002.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q5_K_M/Llama-3.1-70B-Instruct-abliterated-Q5_K_M-00002-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q5_K_M/Llama-3.1-70B-Instruct-lorablated-Q5_K_M-00002-of-00002.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q6_K/Llama-3.1-70B-Instruct-abliterated-Q6_K-00001-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q6_K/Llama-3.1-70B-Instruct-lorablated-Q6_K-00001-of-00002.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q6_K/Llama-3.1-70B-Instruct-abliterated-Q6_K-00002-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q6_K/Llama-3.1-70B-Instruct-lorablated-Q6_K-00002-of-00002.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q8_0/Llama-3.1-70B-Instruct-abliterated-Q8_0-00001-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q8_0/Llama-3.1-70B-Instruct-lorablated-Q8_0-00001-of-00002.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated-Q8_0/Llama-3.1-70B-Instruct-abliterated-Q8_0-00002-of-00002.gguf β†’ Llama-3.1-70B-Instruct-lorablated-Q8_0/Llama-3.1-70B-Instruct-lorablated-Q8_0-00002-of-00002.gguf RENAMED
File without changes
Llama-3.1-70B-Instruct-abliterated.imatrix β†’ Llama-3.1-70B-Instruct-lorablated.imatrix RENAMED
File without changes
README.md CHANGED
@@ -1,20 +1,20 @@
1
  ---
2
- base_model: mlabonne/Llama-3.1-70B-Instruct-abliterated
3
  library_name: transformers
4
  license: llama3.1
5
  pipeline_tag: text-generation
6
  tags:
7
- - abliterated
8
  - uncensored
9
  - mergekit
10
  quantized_by: bartowski
11
  ---
12
 
13
- ## Llamacpp imatrix Quantizations of Llama-3.1-70B-Instruct-abliterated
14
 
15
  Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3496">b3496</a> for quantization.
16
 
17
- Original model: https://huggingface.co/mlabonne/Llama-3.1-70B-Instruct-abliterated
18
 
19
  All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
20
 
@@ -35,6 +35,9 @@ Today Date: 26 Jul 2024
35
 
36
  ```
37
 
 
 
 
38
  ## Download a file (not the whole branch) from below:
39
 
40
  | Filename | Quant type | File Size | Split | Description |
@@ -43,22 +46,22 @@ Today Date: 26 Jul 2024
43
 
44
  | Filename | Quant type | File Size | Split | Description |
45
  | -------- | ---------- | --------- | ----- | ----------- |
46
- | [Llama-3.1-70B-Instruct-abliterated-Q8_0.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/tree/main/Llama-3.1-70B-Instruct-abliterated-Q8_0) | Q8_0 | 74.98GB | true | Extremely high quality, generally unneeded but max available quant. |
47
- | [Llama-3.1-70B-Instruct-abliterated-Q6_K.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/tree/main/Llama-3.1-70B-Instruct-abliterated-Q6_K) | Q6_K | 57.89GB | true | Very high quality, near perfect, *recommended*. |
48
- | [Llama-3.1-70B-Instruct-abliterated-Q5_K_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/tree/main/Llama-3.1-70B-Instruct-abliterated-Q5_K_M) | Q5_K_M | 49.95GB | true | High quality, *recommended*. |
49
- | [Llama-3.1-70B-Instruct-abliterated-Q4_K_L.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-Q4_K_L.gguf) | Q4_K_L | 43.30GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
50
- | [Llama-3.1-70B-Instruct-abliterated-Q4_K_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-Q4_K_M.gguf) | Q4_K_M | 42.52GB | false | Good quality, default size for must use cases, *recommended*. |
51
- | [Llama-3.1-70B-Instruct-abliterated-Q3_K_XL.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-Q3_K_XL.gguf) | Q3_K_XL | 38.06GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
52
- | [Llama-3.1-70B-Instruct-abliterated-IQ4_XS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-IQ4_XS.gguf) | IQ4_XS | 37.90GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
53
- | [Llama-3.1-70B-Instruct-abliterated-Q3_K_L.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-Q3_K_L.gguf) | Q3_K_L | 37.14GB | false | Lower quality but usable, good for low RAM availability. |
54
- | [Llama-3.1-70B-Instruct-abliterated-Q3_K_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-Q3_K_M.gguf) | Q3_K_M | 34.27GB | false | Low quality. |
55
- | [Llama-3.1-70B-Instruct-abliterated-IQ3_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-IQ3_M.gguf) | IQ3_M | 31.94GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
56
- | [Llama-3.1-70B-Instruct-abliterated-IQ3_XXS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-IQ3_XXS.gguf) | IQ3_XXS | 27.47GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
57
- | [Llama-3.1-70B-Instruct-abliterated-Q2_K_L.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-Q2_K_L.gguf) | Q2_K_L | 27.40GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
58
- | [Llama-3.1-70B-Instruct-abliterated-Q2_K.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-Q2_K.gguf) | Q2_K | 26.38GB | false | Very low quality but surprisingly usable. |
59
- | [Llama-3.1-70B-Instruct-abliterated-IQ2_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-IQ2_M.gguf) | IQ2_M | 24.12GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
60
- | [Llama-3.1-70B-Instruct-abliterated-IQ2_XS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-IQ2_XS.gguf) | IQ2_XS | 21.14GB | false | Low quality, uses SOTA techniques to be usable. |
61
- | [Llama-3.1-70B-Instruct-abliterated-IQ2_XXS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF/blob/main/Llama-3.1-70B-Instruct-abliterated-IQ2_XXS.gguf) | IQ2_XXS | 19.10GB | false | Very low quality, uses SOTA techniques to be usable. |
62
 
63
  ## Embed/output weights
64
 
@@ -85,16 +88,16 @@ pip install -U "huggingface_hub[cli]"
85
  Then, you can target the specific file you want:
86
 
87
  ```
88
- huggingface-cli download bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF --include "Llama-3.1-70B-Instruct-abliterated-Q4_K_M.gguf" --local-dir ./
89
  ```
90
 
91
  If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
92
 
93
  ```
94
- huggingface-cli download bartowski/Llama-3.1-70B-Instruct-abliterated-GGUF --include "Llama-3.1-70B-Instruct-abliterated-Q8_0/*" --local-dir ./
95
  ```
96
 
97
- You can either specify a new local-dir (Llama-3.1-70B-Instruct-abliterated-Q8_0) or download them all in place (./)
98
 
99
  ## Which file should I choose?
100
 
 
1
  ---
2
+ base_model: mlabonne/Llama-3.1-70B-Instruct-lorablated
3
  library_name: transformers
4
  license: llama3.1
5
  pipeline_tag: text-generation
6
  tags:
7
+ - lorablated
8
  - uncensored
9
  - mergekit
10
  quantized_by: bartowski
11
  ---
12
 
13
+ ## Llamacpp imatrix Quantizations of Llama-3.1-70B-Instruct-lorablated
14
 
15
  Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3496">b3496</a> for quantization.
16
 
17
+ Original model: https://huggingface.co/mlabonne/Llama-3.1-70B-Instruct-lorablated
18
 
19
  All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
20
 
 
35
 
36
  ```
37
 
38
+ What's new:
39
+ - Just renaming files since mlabonne renamed his
40
+
41
  ## Download a file (not the whole branch) from below:
42
 
43
  | Filename | Quant type | File Size | Split | Description |
 
46
 
47
  | Filename | Quant type | File Size | Split | Description |
48
  | -------- | ---------- | --------- | ----- | ----------- |
49
+ | [Llama-3.1-70B-Instruct-lorablated-Q8_0.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/tree/main/Llama-3.1-70B-Instruct-lorablated-Q8_0) | Q8_0 | 74.98GB | true | Extremely high quality, generally unneeded but max available quant. |
50
+ | [Llama-3.1-70B-Instruct-lorablated-Q6_K.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/tree/main/Llama-3.1-70B-Instruct-lorablated-Q6_K) | Q6_K | 57.89GB | true | Very high quality, near perfect, *recommended*. |
51
+ | [Llama-3.1-70B-Instruct-lorablated-Q5_K_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/tree/main/Llama-3.1-70B-Instruct-lorablated-Q5_K_M) | Q5_K_M | 49.95GB | true | High quality, *recommended*. |
52
+ | [Llama-3.1-70B-Instruct-lorablated-Q4_K_L.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q4_K_L.gguf) | Q4_K_L | 43.30GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
53
+ | [Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf) | Q4_K_M | 42.52GB | false | Good quality, default size for must use cases, *recommended*. |
54
+ | [Llama-3.1-70B-Instruct-lorablated-Q3_K_XL.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q3_K_XL.gguf) | Q3_K_XL | 38.06GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
55
+ | [Llama-3.1-70B-Instruct-lorablated-IQ4_XS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ4_XS.gguf) | IQ4_XS | 37.90GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
56
+ | [Llama-3.1-70B-Instruct-lorablated-Q3_K_L.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q3_K_L.gguf) | Q3_K_L | 37.14GB | false | Lower quality but usable, good for low RAM availability. |
57
+ | [Llama-3.1-70B-Instruct-lorablated-Q3_K_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q3_K_M.gguf) | Q3_K_M | 34.27GB | false | Low quality. |
58
+ | [Llama-3.1-70B-Instruct-lorablated-IQ3_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ3_M.gguf) | IQ3_M | 31.94GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
59
+ | [Llama-3.1-70B-Instruct-lorablated-IQ3_XXS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ3_XXS.gguf) | IQ3_XXS | 27.47GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
60
+ | [Llama-3.1-70B-Instruct-lorablated-Q2_K_L.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q2_K_L.gguf) | Q2_K_L | 27.40GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
61
+ | [Llama-3.1-70B-Instruct-lorablated-Q2_K.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q2_K.gguf) | Q2_K | 26.38GB | false | Very low quality but surprisingly usable. |
62
+ | [Llama-3.1-70B-Instruct-lorablated-IQ2_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ2_M.gguf) | IQ2_M | 24.12GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
63
+ | [Llama-3.1-70B-Instruct-lorablated-IQ2_XS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ2_XS.gguf) | IQ2_XS | 21.14GB | false | Low quality, uses SOTA techniques to be usable. |
64
+ | [Llama-3.1-70B-Instruct-lorablated-IQ2_XXS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ2_XXS.gguf) | IQ2_XXS | 19.10GB | false | Very low quality, uses SOTA techniques to be usable. |
65
 
66
  ## Embed/output weights
67
 
 
88
  Then, you can target the specific file you want:
89
 
90
  ```
91
+ huggingface-cli download bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF --include "Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf" --local-dir ./
92
  ```
93
 
94
  If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
95
 
96
  ```
97
+ huggingface-cli download bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF --include "Llama-3.1-70B-Instruct-lorablated-Q8_0/*" --local-dir ./
98
  ```
99
 
100
+ You can either specify a new local-dir (Llama-3.1-70B-Instruct-lorablated-Q8_0) or download them all in place (./)
101
 
102
  ## Which file should I choose?
103