Rename files to match mlabonne
Browse files- Llama-3.1-70B-Instruct-abliterated-IQ2_M.gguf β Llama-3.1-70B-Instruct-lorablated-IQ2_M.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-IQ2_XS.gguf β Llama-3.1-70B-Instruct-lorablated-IQ2_XS.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-IQ2_XXS.gguf β Llama-3.1-70B-Instruct-lorablated-IQ2_XXS.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-IQ3_M.gguf β Llama-3.1-70B-Instruct-lorablated-IQ3_M.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-IQ3_XXS.gguf β Llama-3.1-70B-Instruct-lorablated-IQ3_XXS.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-IQ4_XS.gguf β Llama-3.1-70B-Instruct-lorablated-IQ4_XS.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q2_K.gguf β Llama-3.1-70B-Instruct-lorablated-Q2_K.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q2_K_L.gguf β Llama-3.1-70B-Instruct-lorablated-Q2_K_L.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q3_K_L.gguf β Llama-3.1-70B-Instruct-lorablated-Q3_K_L.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q3_K_M.gguf β Llama-3.1-70B-Instruct-lorablated-Q3_K_M.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q3_K_XL.gguf β Llama-3.1-70B-Instruct-lorablated-Q3_K_XL.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q4_K_L.gguf β Llama-3.1-70B-Instruct-lorablated-Q4_K_L.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q4_K_M.gguf β Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q5_K_M/Llama-3.1-70B-Instruct-abliterated-Q5_K_M-00001-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q5_K_M/Llama-3.1-70B-Instruct-lorablated-Q5_K_M-00001-of-00002.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q5_K_M/Llama-3.1-70B-Instruct-abliterated-Q5_K_M-00002-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q5_K_M/Llama-3.1-70B-Instruct-lorablated-Q5_K_M-00002-of-00002.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q6_K/Llama-3.1-70B-Instruct-abliterated-Q6_K-00001-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q6_K/Llama-3.1-70B-Instruct-lorablated-Q6_K-00001-of-00002.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q6_K/Llama-3.1-70B-Instruct-abliterated-Q6_K-00002-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q6_K/Llama-3.1-70B-Instruct-lorablated-Q6_K-00002-of-00002.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q8_0/Llama-3.1-70B-Instruct-abliterated-Q8_0-00001-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q8_0/Llama-3.1-70B-Instruct-lorablated-Q8_0-00001-of-00002.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated-Q8_0/Llama-3.1-70B-Instruct-abliterated-Q8_0-00002-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q8_0/Llama-3.1-70B-Instruct-lorablated-Q8_0-00002-of-00002.gguf +0 -0
- Llama-3.1-70B-Instruct-abliterated.imatrix β Llama-3.1-70B-Instruct-lorablated.imatrix +0 -0
- README.md +26 -23
Llama-3.1-70B-Instruct-abliterated-IQ2_M.gguf β Llama-3.1-70B-Instruct-lorablated-IQ2_M.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-IQ2_XS.gguf β Llama-3.1-70B-Instruct-lorablated-IQ2_XS.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-IQ2_XXS.gguf β Llama-3.1-70B-Instruct-lorablated-IQ2_XXS.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-IQ3_M.gguf β Llama-3.1-70B-Instruct-lorablated-IQ3_M.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-IQ3_XXS.gguf β Llama-3.1-70B-Instruct-lorablated-IQ3_XXS.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-IQ4_XS.gguf β Llama-3.1-70B-Instruct-lorablated-IQ4_XS.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q2_K.gguf β Llama-3.1-70B-Instruct-lorablated-Q2_K.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q2_K_L.gguf β Llama-3.1-70B-Instruct-lorablated-Q2_K_L.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q3_K_L.gguf β Llama-3.1-70B-Instruct-lorablated-Q3_K_L.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q3_K_M.gguf β Llama-3.1-70B-Instruct-lorablated-Q3_K_M.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q3_K_XL.gguf β Llama-3.1-70B-Instruct-lorablated-Q3_K_XL.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q4_K_L.gguf β Llama-3.1-70B-Instruct-lorablated-Q4_K_L.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q4_K_M.gguf β Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q5_K_M/Llama-3.1-70B-Instruct-abliterated-Q5_K_M-00001-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q5_K_M/Llama-3.1-70B-Instruct-lorablated-Q5_K_M-00001-of-00002.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q5_K_M/Llama-3.1-70B-Instruct-abliterated-Q5_K_M-00002-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q5_K_M/Llama-3.1-70B-Instruct-lorablated-Q5_K_M-00002-of-00002.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q6_K/Llama-3.1-70B-Instruct-abliterated-Q6_K-00001-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q6_K/Llama-3.1-70B-Instruct-lorablated-Q6_K-00001-of-00002.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q6_K/Llama-3.1-70B-Instruct-abliterated-Q6_K-00002-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q6_K/Llama-3.1-70B-Instruct-lorablated-Q6_K-00002-of-00002.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q8_0/Llama-3.1-70B-Instruct-abliterated-Q8_0-00001-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q8_0/Llama-3.1-70B-Instruct-lorablated-Q8_0-00001-of-00002.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated-Q8_0/Llama-3.1-70B-Instruct-abliterated-Q8_0-00002-of-00002.gguf β Llama-3.1-70B-Instruct-lorablated-Q8_0/Llama-3.1-70B-Instruct-lorablated-Q8_0-00002-of-00002.gguf
RENAMED
File without changes
|
Llama-3.1-70B-Instruct-abliterated.imatrix β Llama-3.1-70B-Instruct-lorablated.imatrix
RENAMED
File without changes
|
README.md
CHANGED
@@ -1,20 +1,20 @@
|
|
1 |
---
|
2 |
-
base_model: mlabonne/Llama-3.1-70B-Instruct-
|
3 |
library_name: transformers
|
4 |
license: llama3.1
|
5 |
pipeline_tag: text-generation
|
6 |
tags:
|
7 |
-
-
|
8 |
- uncensored
|
9 |
- mergekit
|
10 |
quantized_by: bartowski
|
11 |
---
|
12 |
|
13 |
-
## Llamacpp imatrix Quantizations of Llama-3.1-70B-Instruct-
|
14 |
|
15 |
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3496">b3496</a> for quantization.
|
16 |
|
17 |
-
Original model: https://huggingface.co/mlabonne/Llama-3.1-70B-Instruct-
|
18 |
|
19 |
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
|
20 |
|
@@ -35,6 +35,9 @@ Today Date: 26 Jul 2024
|
|
35 |
|
36 |
```
|
37 |
|
|
|
|
|
|
|
38 |
## Download a file (not the whole branch) from below:
|
39 |
|
40 |
| Filename | Quant type | File Size | Split | Description |
|
@@ -43,22 +46,22 @@ Today Date: 26 Jul 2024
|
|
43 |
|
44 |
| Filename | Quant type | File Size | Split | Description |
|
45 |
| -------- | ---------- | --------- | ----- | ----------- |
|
46 |
-
| [Llama-3.1-70B-Instruct-
|
47 |
-
| [Llama-3.1-70B-Instruct-
|
48 |
-
| [Llama-3.1-70B-Instruct-
|
49 |
-
| [Llama-3.1-70B-Instruct-
|
50 |
-
| [Llama-3.1-70B-Instruct-
|
51 |
-
| [Llama-3.1-70B-Instruct-
|
52 |
-
| [Llama-3.1-70B-Instruct-
|
53 |
-
| [Llama-3.1-70B-Instruct-
|
54 |
-
| [Llama-3.1-70B-Instruct-
|
55 |
-
| [Llama-3.1-70B-Instruct-
|
56 |
-
| [Llama-3.1-70B-Instruct-
|
57 |
-
| [Llama-3.1-70B-Instruct-
|
58 |
-
| [Llama-3.1-70B-Instruct-
|
59 |
-
| [Llama-3.1-70B-Instruct-
|
60 |
-
| [Llama-3.1-70B-Instruct-
|
61 |
-
| [Llama-3.1-70B-Instruct-
|
62 |
|
63 |
## Embed/output weights
|
64 |
|
@@ -85,16 +88,16 @@ pip install -U "huggingface_hub[cli]"
|
|
85 |
Then, you can target the specific file you want:
|
86 |
|
87 |
```
|
88 |
-
huggingface-cli download bartowski/Llama-3.1-70B-Instruct-
|
89 |
```
|
90 |
|
91 |
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
|
92 |
|
93 |
```
|
94 |
-
huggingface-cli download bartowski/Llama-3.1-70B-Instruct-
|
95 |
```
|
96 |
|
97 |
-
You can either specify a new local-dir (Llama-3.1-70B-Instruct-
|
98 |
|
99 |
## Which file should I choose?
|
100 |
|
|
|
1 |
---
|
2 |
+
base_model: mlabonne/Llama-3.1-70B-Instruct-lorablated
|
3 |
library_name: transformers
|
4 |
license: llama3.1
|
5 |
pipeline_tag: text-generation
|
6 |
tags:
|
7 |
+
- lorablated
|
8 |
- uncensored
|
9 |
- mergekit
|
10 |
quantized_by: bartowski
|
11 |
---
|
12 |
|
13 |
+
## Llamacpp imatrix Quantizations of Llama-3.1-70B-Instruct-lorablated
|
14 |
|
15 |
Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3496">b3496</a> for quantization.
|
16 |
|
17 |
+
Original model: https://huggingface.co/mlabonne/Llama-3.1-70B-Instruct-lorablated
|
18 |
|
19 |
All quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)
|
20 |
|
|
|
35 |
|
36 |
```
|
37 |
|
38 |
+
What's new:
|
39 |
+
- Just renaming files since mlabonne renamed his
|
40 |
+
|
41 |
## Download a file (not the whole branch) from below:
|
42 |
|
43 |
| Filename | Quant type | File Size | Split | Description |
|
|
|
46 |
|
47 |
| Filename | Quant type | File Size | Split | Description |
|
48 |
| -------- | ---------- | --------- | ----- | ----------- |
|
49 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q8_0.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/tree/main/Llama-3.1-70B-Instruct-lorablated-Q8_0) | Q8_0 | 74.98GB | true | Extremely high quality, generally unneeded but max available quant. |
|
50 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q6_K.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/tree/main/Llama-3.1-70B-Instruct-lorablated-Q6_K) | Q6_K | 57.89GB | true | Very high quality, near perfect, *recommended*. |
|
51 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q5_K_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/tree/main/Llama-3.1-70B-Instruct-lorablated-Q5_K_M) | Q5_K_M | 49.95GB | true | High quality, *recommended*. |
|
52 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q4_K_L.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q4_K_L.gguf) | Q4_K_L | 43.30GB | false | Uses Q8_0 for embed and output weights. Good quality, *recommended*. |
|
53 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf) | Q4_K_M | 42.52GB | false | Good quality, default size for must use cases, *recommended*. |
|
54 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q3_K_XL.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q3_K_XL.gguf) | Q3_K_XL | 38.06GB | false | Uses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability. |
|
55 |
+
| [Llama-3.1-70B-Instruct-lorablated-IQ4_XS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ4_XS.gguf) | IQ4_XS | 37.90GB | false | Decent quality, smaller than Q4_K_S with similar performance, *recommended*. |
|
56 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q3_K_L.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q3_K_L.gguf) | Q3_K_L | 37.14GB | false | Lower quality but usable, good for low RAM availability. |
|
57 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q3_K_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q3_K_M.gguf) | Q3_K_M | 34.27GB | false | Low quality. |
|
58 |
+
| [Llama-3.1-70B-Instruct-lorablated-IQ3_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ3_M.gguf) | IQ3_M | 31.94GB | false | Medium-low quality, new method with decent performance comparable to Q3_K_M. |
|
59 |
+
| [Llama-3.1-70B-Instruct-lorablated-IQ3_XXS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ3_XXS.gguf) | IQ3_XXS | 27.47GB | false | Lower quality, new method with decent performance, comparable to Q3 quants. |
|
60 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q2_K_L.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q2_K_L.gguf) | Q2_K_L | 27.40GB | false | Uses Q8_0 for embed and output weights. Very low quality but surprisingly usable. |
|
61 |
+
| [Llama-3.1-70B-Instruct-lorablated-Q2_K.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-Q2_K.gguf) | Q2_K | 26.38GB | false | Very low quality but surprisingly usable. |
|
62 |
+
| [Llama-3.1-70B-Instruct-lorablated-IQ2_M.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ2_M.gguf) | IQ2_M | 24.12GB | false | Relatively low quality, uses SOTA techniques to be surprisingly usable. |
|
63 |
+
| [Llama-3.1-70B-Instruct-lorablated-IQ2_XS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ2_XS.gguf) | IQ2_XS | 21.14GB | false | Low quality, uses SOTA techniques to be usable. |
|
64 |
+
| [Llama-3.1-70B-Instruct-lorablated-IQ2_XXS.gguf](https://huggingface.co/bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF/blob/main/Llama-3.1-70B-Instruct-lorablated-IQ2_XXS.gguf) | IQ2_XXS | 19.10GB | false | Very low quality, uses SOTA techniques to be usable. |
|
65 |
|
66 |
## Embed/output weights
|
67 |
|
|
|
88 |
Then, you can target the specific file you want:
|
89 |
|
90 |
```
|
91 |
+
huggingface-cli download bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF --include "Llama-3.1-70B-Instruct-lorablated-Q4_K_M.gguf" --local-dir ./
|
92 |
```
|
93 |
|
94 |
If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
|
95 |
|
96 |
```
|
97 |
+
huggingface-cli download bartowski/Llama-3.1-70B-Instruct-lorablated-GGUF --include "Llama-3.1-70B-Instruct-lorablated-Q8_0/*" --local-dir ./
|
98 |
```
|
99 |
|
100 |
+
You can either specify a new local-dir (Llama-3.1-70B-Instruct-lorablated-Q8_0) or download them all in place (./)
|
101 |
|
102 |
## Which file should I choose?
|
103 |
|