jacobfulano
commited on
Commit
•
8c92147
1
Parent(s):
c271818
Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,7 @@ tags:
|
|
11 |
# MPT-7B-Instruct
|
12 |
|
13 |
MPT-7B-Instruct is a model for short-form instruction following.
|
14 |
-
It is built by finetuning [MPT-7B
|
15 |
* License: _CC-By-SA-3.0_ (commercial use permitted)
|
16 |
* [Online Demo](https://huggingface.co/spaces/mosaicml/mpt-7b-instruct)
|
17 |
|
@@ -99,10 +99,30 @@ For more details on the pretraining process, see [MPT-7B](https://huggingface.co
|
|
99 |
|
100 |
The data was tokenized using the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
|
101 |
|
102 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
103 |
|
104 |
-
This model was finetuned on 440 A100-40GBs for about half a day using the [MosaicML Platform](https://www.mosaicml.com/platform).
|
105 |
|
106 |
## Acknowledgements
|
107 |
|
108 |
-
This model was finetuned by Sam Havens and the MosaicML NLP team
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
# MPT-7B-Instruct
|
12 |
|
13 |
MPT-7B-Instruct is a model for short-form instruction following.
|
14 |
+
It is built by finetuning [MPT-7B](https://huggingface.co/spaces/mosaicml/mpt-7b) on a [dataset](https://huggingface.co/datasets/sam-mosaic/dolly_hhrlhf) derived from the [Databricks Dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) and the [Anthropic Helpful and Harmless (HH-RLHF)](https://huggingface.co/datasets/Anthropic/hh-rlhf) datasets.
|
15 |
* License: _CC-By-SA-3.0_ (commercial use permitted)
|
16 |
* [Online Demo](https://huggingface.co/spaces/mosaicml/mpt-7b-instruct)
|
17 |
|
|
|
99 |
|
100 |
The data was tokenized using the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
|
101 |
|
102 |
+
## Limitations and Biases
|
103 |
+
|
104 |
+
_The following language is modified from [EleutherAI's GPT-NeoX-20B](https://huggingface.co/EleutherAI/gpt-neox-20b)_
|
105 |
+
|
106 |
+
MPT-7B-Chat can produce factually incorrect output, and should not be relied on to produce factually accurate information.
|
107 |
+
MPT-7B-CHat was trained on various public datasets.
|
108 |
+
While great efforts have been taken to clean the pretraining data, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
|
109 |
|
|
|
110 |
|
111 |
## Acknowledgements
|
112 |
|
113 |
+
This model was finetuned by Sam Havens and the MosaicML NLP team
|
114 |
+
|
115 |
+
## Citation
|
116 |
+
|
117 |
+
Please cite this model using the following format:
|
118 |
+
|
119 |
+
```
|
120 |
+
@online{MosaicML2023Introducing,
|
121 |
+
author = {MosaicML NLP Team},
|
122 |
+
title = {Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs},
|
123 |
+
year = {2023},
|
124 |
+
url = {www.mosaicml.com/blog/mpt-7b},
|
125 |
+
note = {Accessed: 2023-03-28}, % change this date
|
126 |
+
urldate = {2023-03-28} % change this date
|
127 |
+
}
|
128 |
+
```
|