Upload README.md
Browse files
README.md
CHANGED
@@ -7,7 +7,7 @@ language:
|
|
7 |
- en
|
8 |
library_name: transformers
|
9 |
license: apache-2.0
|
10 |
-
model_creator:
|
11 |
model_name: OpenInstruct Mistral 7B
|
12 |
model_type: mistral
|
13 |
pipeline_tag: text-generation
|
@@ -45,13 +45,13 @@ quantized_by: TheBloke
|
|
45 |
<!-- header end -->
|
46 |
|
47 |
# OpenInstruct Mistral 7B - AWQ
|
48 |
-
- Model creator: [
|
49 |
- Original model: [OpenInstruct Mistral 7B](https://huggingface.co/monology/openinstruct-mistral-7b)
|
50 |
|
51 |
<!-- description start -->
|
52 |
## Description
|
53 |
|
54 |
-
This repo contains AWQ model files for [
|
55 |
|
56 |
These files were quantised using hardware kindly provided by [Massed Compute](https://massedcompute.com/).
|
57 |
|
@@ -75,7 +75,7 @@ It is supported by:
|
|
75 |
* [AWQ model(s) for GPU inference.](https://huggingface.co/TheBloke/openinstruct-mistral-7B-AWQ)
|
76 |
* [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/openinstruct-mistral-7B-GPTQ)
|
77 |
* [2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference](https://huggingface.co/TheBloke/openinstruct-mistral-7B-GGUF)
|
78 |
-
* [
|
79 |
<!-- repositories-available end -->
|
80 |
|
81 |
<!-- prompt-template start -->
|
@@ -103,7 +103,7 @@ Models are released as sharded safetensors files.
|
|
103 |
|
104 |
| Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
|
105 |
| ------ | ---- | -- | ----------- | ------- | ---- |
|
106 |
-
| [main](https://huggingface.co/TheBloke/openinstruct-mistral-7B-AWQ/tree/main) | 4 | 128 | [
|
107 |
|
108 |
<!-- README_AWQ.md-provided-files end -->
|
109 |
|
@@ -372,7 +372,7 @@ And thank you again to a16z for their generous grant.
|
|
372 |
|
373 |
<!-- footer end -->
|
374 |
|
375 |
-
# Original model card:
|
376 |
|
377 |
|
378 |
# OpenInstruct Mistral-7B
|
@@ -383,7 +383,7 @@ Quantized to FP16 and released under the [Apache-2.0](https://choosealicense.com
|
|
383 |
Compute generously provided by [Higgsfield AI](https://higgsfield.ai/model/655559e6b5777dab620095e0).
|
384 |
|
385 |
|
386 |
-
Prompt format: Alpaca
|
387 |
```
|
388 |
Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
389 |
|
@@ -393,4 +393,10 @@ Below is an instruction that describes a task. Write a response that appropriate
|
|
393 |
### Response:
|
394 |
```
|
395 |
|
|
|
|
|
|
|
|
|
|
|
|
|
396 |
\*as of 21 Nov 2023. "commercially-usable" includes both an open-source base model and a *non-synthetic* open-source finetune dataset. updated leaderboard results available [here](https://huggingfaceh4-open-llm-leaderboard.hf.space).
|
|
|
7 |
- en
|
8 |
library_name: transformers
|
9 |
license: apache-2.0
|
10 |
+
model_creator: Devin Gulliver
|
11 |
model_name: OpenInstruct Mistral 7B
|
12 |
model_type: mistral
|
13 |
pipeline_tag: text-generation
|
|
|
45 |
<!-- header end -->
|
46 |
|
47 |
# OpenInstruct Mistral 7B - AWQ
|
48 |
+
- Model creator: [Devin Gulliver](https://huggingface.co/monology)
|
49 |
- Original model: [OpenInstruct Mistral 7B](https://huggingface.co/monology/openinstruct-mistral-7b)
|
50 |
|
51 |
<!-- description start -->
|
52 |
## Description
|
53 |
|
54 |
+
This repo contains AWQ model files for [Devin Gulliver's OpenInstruct Mistral 7B](https://huggingface.co/monology/openinstruct-mistral-7b).
|
55 |
|
56 |
These files were quantised using hardware kindly provided by [Massed Compute](https://massedcompute.com/).
|
57 |
|
|
|
75 |
* [AWQ model(s) for GPU inference.](https://huggingface.co/TheBloke/openinstruct-mistral-7B-AWQ)
|
76 |
* [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/openinstruct-mistral-7B-GPTQ)
|
77 |
* [2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference](https://huggingface.co/TheBloke/openinstruct-mistral-7B-GGUF)
|
78 |
+
* [Devin Gulliver's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/monology/openinstruct-mistral-7b)
|
79 |
<!-- repositories-available end -->
|
80 |
|
81 |
<!-- prompt-template start -->
|
|
|
103 |
|
104 |
| Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
|
105 |
| ------ | ---- | -- | ----------- | ------- | ---- |
|
106 |
+
| [main](https://huggingface.co/TheBloke/openinstruct-mistral-7B-AWQ/tree/main) | 4 | 128 | [VMware Open Instruct](https://huggingface.co/datasets/VMware/open-instruct/viewer/) | 4096 | 4.15 GB
|
107 |
|
108 |
<!-- README_AWQ.md-provided-files end -->
|
109 |
|
|
|
372 |
|
373 |
<!-- footer end -->
|
374 |
|
375 |
+
# Original model card: Devin Gulliver's OpenInstruct Mistral 7B
|
376 |
|
377 |
|
378 |
# OpenInstruct Mistral-7B
|
|
|
383 |
Compute generously provided by [Higgsfield AI](https://higgsfield.ai/model/655559e6b5777dab620095e0).
|
384 |
|
385 |
|
386 |
+
## Prompt format: Alpaca
|
387 |
```
|
388 |
Below is an instruction that describes a task. Write a response that appropriately completes the request.
|
389 |
|
|
|
393 |
### Response:
|
394 |
```
|
395 |
|
396 |
+
## Recommended preset:
|
397 |
+
- temperature: 0.2
|
398 |
+
- top_k: 50
|
399 |
+
- top_p 0.95
|
400 |
+
- repetition_penalty: 1.1
|
401 |
+
|
402 |
\*as of 21 Nov 2023. "commercially-usable" includes both an open-source base model and a *non-synthetic* open-source finetune dataset. updated leaderboard results available [here](https://huggingfaceh4-open-llm-leaderboard.hf.space).
|