Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+gpt-neox-20b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,76 @@
+---
+language:
+- en
+tags:
+- pytorch
+- causal-lm
+- TensorBlock
+- GGUF
+license: apache-2.0
+datasets:
+- EleutherAI/pile
+base_model: EleutherAI/gpt-neox-20b
+---
+
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+<div style="display: flex; justify-content: space-between; width: 100%;">
+    <div style="display: flex; flex-direction: column; align-items: flex-start;">
+        <p style="margin-top: 0.5em; margin-bottom: 0em;">
+            Feedback and support: TensorBlock's  <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
+        </p>
+    </div>
+</div>
+
+## EleutherAI/gpt-neox-20b - GGUF
+
+This repo contains GGUF format model files for [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b).
+
+The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
+
+## Prompt template
+
+```
+
+```
+
+## Model file specification
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| [gpt-neox-20b-Q2_K.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q2_K.gguf) | Q2_K | 7.223 GB | smallest, significant quality loss - not recommended for most purposes |
+| [gpt-neox-20b-Q3_K_S.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q3_K_S.gguf) | Q3_K_S | 8.352 GB | very small, high quality loss |
+| [gpt-neox-20b-Q3_K_M.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q3_K_M.gguf) | Q3_K_M | 10.030 GB | very small, high quality loss |
+| [gpt-neox-20b-Q3_K_L.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q3_K_L.gguf) | Q3_K_L | 10.962 GB | small, substantial quality loss |
+| [gpt-neox-20b-Q4_0.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q4_0.gguf) | Q4_0 | 10.855 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
+| [gpt-neox-20b-Q4_K_S.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q4_K_S.gguf) | Q4_K_S | 10.943 GB | small, greater quality loss |
+| [gpt-neox-20b-Q4_K_M.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q4_K_M.gguf) | Q4_K_M | 12.233 GB | medium, balanced quality - recommended |
+| [gpt-neox-20b-Q5_0.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q5_0.gguf) | Q5_0 | 13.212 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
+| [gpt-neox-20b-Q5_K_S.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q5_K_S.gguf) | Q5_K_S | 13.212 GB | large, low quality loss - recommended |
+| [gpt-neox-20b-Q5_K_M.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q5_K_M.gguf) | Q5_K_M | 14.239 GB | large, very low quality loss - recommended |
+| [gpt-neox-20b-Q6_K.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q6_K.gguf) | Q6_K | 15.715 GB | very large, extremely low quality loss |
+| [gpt-neox-20b-Q8_0.gguf](https://huggingface.co/tensorblock/gpt-neox-20b-GGUF/tree/main/gpt-neox-20b-Q8_0.gguf) | Q8_0 | 20.351 GB | very large, extremely low quality loss - not recommended |
+
+
+## Downloading instruction
+
+### Command line
+
+Firstly, install Huggingface Client
+
+```shell
+pip install -U "huggingface_hub[cli]"
+```
+
+Then, downoad the individual model file the a local directory
+
+```shell
+huggingface-cli download tensorblock/gpt-neox-20b-GGUF --include "gpt-neox-20b-Q2_K.gguf" --local-dir MY_LOCAL_DIR
+```
+
+If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:
+
+```shell
+huggingface-cli download tensorblock/gpt-neox-20b-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
+```

gpt-neox-20b-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95e6d61a5d591cacf240f23058b2766b7a219a996490ea1d9288a2ead521c03a
+size 7755173248

gpt-neox-20b-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ebae54c27067559bf3274f11a2fbca51f27c9194e87553a3b2a2c9ee57ffb72
+size 11770314112

gpt-neox-20b-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:825e5c9ddf0b5575f7c12354233f004f1882b88858a132c16e54d8ee46c84f01
+size 10769972608

gpt-neox-20b-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1cf85c707289d40006aa20839c556c6b5ce61de9bce2a10bcf69394ba9a21d3
+size 8967470464

gpt-neox-20b-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a46480a22ca6792c15fb9135d87e37d3063ab2c9d7861a1c3ff4c32ea7460be
+size 11655753088

gpt-neox-20b-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0e8751d10e0d0aeeb031f3a83340f4a49771608b3d3ff70e2949f8dea495d35
+size 13135031680

gpt-neox-20b-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70b11ef439b4b9854d28809cd423e650435af5ca4d9d286202992ff7af7d22ed
+size 11750124928

gpt-neox-20b-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0227c6ae51703fa0cc0ed78188a17a201e10c0c4528b8b9f08d19239be999f5
+size 14185901440

gpt-neox-20b-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ac35cd8086b8b7ff526c1f6a6175f0af3f4604f1da820352efc4244d0dcf3ea
+size 15288872320

gpt-neox-20b-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:265b80ce0c156716e75c9176a4d167777665e071508ea30f819f0e4876012ab8
+size 14185901440

gpt-neox-20b-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4f0860bd59536e3eaa20cb1868f4c31a6247b5e9bcdb2827798d5dd8a88dc7a0
+size 16874184064

gpt-neox-20b-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01c310611782aafdc883f82b77f61223d9880f0d116fa15c76a09bfbf6681ed2
+size 21851389312