Upload folder using huggingface_hub

README.md

@@ -0,0 +1,57 @@
+---
+license: apache-2.0
+language:
+  - en
+tags:
+  - ctranslate2
+  - RedPajama-INCITE-7B-Chat
+  - redpajama-chat-7b
+  - quantization
+  - int8
+---
+
+# RedPajama-INCITE-7B-Chat Q8
+
+The model is quantized version of the [togethercomputer/RedPajama-INCITE-7B-Chat](https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Chat) with int8 quantization.
+
+## Model Details
+
+### Model Description
+
+The model being quantized using [CTranslate2](https://opennmt.net/CTranslate2/) with the following command:
+
+```
+ct2-transformers-converter --model togethercomputer/redpajama-chat-7b --output_dir togethercomputer/redpajama-chat-7b-ct2 --copy_files tokenizer.json tokenizer_config.json special_tokens_map.json generation_config.json --quantization int8 --force --low_cpu_mem_usage
+```
+
+If you want to perform the quantization yourself, you need to install the following dependencies:
+
+```
+pip install -qU ctranslate2 transformers[torch] accelerate
+```
+
+- **Shared by:** Lim Chee Kin
+- **License:** Apache 2.0
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+```python
+import ctranslate2
+import transformers
+
+generator = ctranslate2.Generator("limcheekin/redpajama-chat-7b-ct2")
+tokenizer = transformers.AutoTokenizer.from_pretrained("limcheekin/redpajama-chat-7b-ct2")
+
+prompt = "<human>: Who is Alan Turing?\n<bot>:"
+tokens = tokenizer.convert_ids_to_tokens(tokenizer.encode(prompt))
+
+results = generator.generate_batch([tokens], max_length=256, sampling_topk=10)
+
+text = tokenizer.decode(results[0].sequences_ids[0])
+```
+
+The code is taken from https://opennmt.net/CTranslate2/guides/transformers.html#mpt.
+
+The key method of the code above is `generate_batch`, you can find out [its supported parameters here](https://opennmt.net/CTranslate2/python/ctranslate2.Generator.html#ctranslate2.Generator.generate_batch).

config.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "layer_norm_epsilon": null,
+  "unk_token": "<|endoftext|>"
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "transformers_version": "4.29.1"
+}

model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3893f998a30bbb1055f9642a95c5da5e4068f603ae29e2141ad01f4256733fdf
+size 6867595176

special_tokens_map.json

@@ -0,0 +1,5 @@
+{
+  "bos_token": "<|endoftext|>",
+  "eos_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,9 @@
+{
+  "add_prefix_space": false,
+  "bos_token": "<|endoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 2048,
+  "tokenizer_class": "GPTNeoXTokenizer",
+  "unk_token": "<|endoftext|>"
+}